HtmlUnit正在抛出内存并可能泄漏内存 [英] HtmlUnit is throwing Out Of Memory and maybe leaking memory

问题描述

我在启用了javascript的情况下使用Selenium和HtmlUnitDriver，我得到Out Of Memory错误（我使用Java）。我只是浏览同一页面。我只使用一个GET命令。哪种解决办法可以克服这种情况？

I use Selenium with HtmlUnitDriver with javascript enabled and I get Out Of Memory errors (I use Java). I just browse the same page. I am only using a single GET command. Which is the solution to overcome the situation?

推荐答案

我遇到了类似的问题。它最终成为自动加载帧的问题......一个无法禁用的功能。

I've had a similar issue. It ended up being an issue with auto-loading of frames... a feature that can't be disabled.

看看这个：极其简单的代码无法在HtmlUnit中使用

这可能会有所帮助。

更新

HtmlUnit的当前版本是2.10。我开始在2.8版本中使用HtmlUnit，每个新版本最终都会占用更多内存。我得到了一个点，其中启用javascript获取5个页面导致2GB的进程。

Current version of HtmlUnit is 2.10. I started using HtmlUnit from version 2.8 and each new version ended up eating more memory. I got to a point in which fetching 5 pages with javascript enabled resulted in a process of 2GB.

从javascript的角度来看，有很多方法可以改善这种情况。但是，当您无法修改javascript（例如：如果您正在抓取某个网站）时，您的双手将被绑定。当然，禁用javascript是最好的方法。但是，这可能会导致获取的页面与预期的页面不同。

There are many ways to improve this situation from a javascript point of view. However, when you can't modify the javascript (eg: if you are crawling a site) your hands are tied. Disabling javascript is, of course, the best way to go. However, this might result in fetched pages being different from the expected ones.

但我确实设法克服了这种情况。经过多次测试，我注意到它可能不是HtmlUnit的问题（我认为从一开始就是有罪的）。它似乎是JVM。从Sun的JVM更改为OpenJDK可以解决这个问题，现在这个过程不需要200GB，而是需要200GB内存。我正在添加版本信息。

I did manage to overcome this situation, though. After many tests, I noticed that it might not be an issue with HtmlUnit (which I thought was the guilty one from the beginning). It seemed to be the JVM. Changing from Sun's JVM to OpenJDK did the trick and now the process instead of eating 2GB of memory only requires 200MB. I'm adding version information.

Sun的（Oracle）32位JVM：

Sun's (Oracle) 32-bit JVM:

$java -version
java version "1.6.0.26"
Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
Java HotSpot(TM) Server VM (build 20.1-b02, mixed mode)

OpenJDK 32位JVM：

OpenJDK 32-bit JVM:

$java -version
java version "1.6.0_18"
OpenJDK Runtime Environment (IcedTea6 1.8.13) (6b18-1.8.13-0+squeeze2)
OpenJDK Server VM (build 14.0-b16, mixed mode)

操作系统：

$ uname -a
Linux vostro1015 2.6.32-5-686-bigmem #1 SMP Sun May 6 04:39:05 UTC 2012 i686 GNU/Linux

请分享您的经验这个。

这篇关于HtmlUnit正在抛出内存并可能泄漏内存的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！