我的代码中的内存不足异常 [英] Out of memory exception in my code

问题描述

作为对Oracle数据库的压力测试的一部分，我正在长时间运行代码，并使用Java版本"1.4.2".简而言之，我正在做的是:

while(true)
{
    Allocating some memory as a blob
    byte[] data = new byte[1000];
    stmt = fConnection.prepareStatement(query); // [compiling an insert query which uses the above blob]
    stmt.execute();  // I insert this blob-row in the database. 
stmt.close();

}

现在，我想将此测试运行8-10个小时.但是，显然在插入约1500万条记录后，我遇到了 java.lang.OutOfMemoryError

我正在使用-Xms512m -Xmx2g运行它.我尝试使用更高的值，但我似乎没有那么多硬件，我也不认为这是必需的:

    java -Xms512m -Xmx4g -jar XX.jar
    Invalid maximum heap size: -Xmx4g
    The specified size exceeds the maximum representable size.
    Could not create the Java virtual machine.
    java -Xms2g -Xmx3g -jar XX.jar
    Error occurred during initialization of VM
    Could not reserve enough space for object heap
    Could not create the Java virtual machine.

我将其作为多线程程序运行.因此，有10个线程正在执行插入操作.

有没有什么办法可以使我的程序免受黑客攻击.我的意思是，如果我决定将其运行15到20个小时而不是8到10个小时，该怎么办?

添加了stmt.close，因为我已经在代码中使用了它. 根据评论进行一些更改

谢谢

P.S:抱歉，无法发布NDA的代码bec

解决方案

基本上，我认为您在吠叫错误的树:

• JVM/GC 将设法解除分配无法访问的对象，无论您以多快的速度分配它们.如果您正在运行经典的非并发GC，那么JVM只会停止做其他事情，直到GC释放了内存.如果您将JVM配置为使用并发GC，它将尝试同时运行GC和普通工作线程...并在无法跟上时恢复为停止所有操作并收集"行为.

• 如果内存不足，那是因为您的应用程序(或其正在使用的库/驱动程序)中的某些内容正在泄漏内存.换句话说，即使您的应用程序不再需要对象，某些原因也会导致对象保持可访问性.

正如评论所指出的那样，您需要使用内存探查器/堆转储有条不紊地解决此问题.随机更改或归咎于GC不太可能解决问题.

(当您说"......我一直都在使用stmt.close()" 时，我认为这意味着您的代码看起来像这样:

    PreparedStatement stmt = ... 
    try {
        stmt.execute();
        // ...
    } finally {
        stmt.close();
    }

如果不将close呼叫放入finally中，则可能每次都没有呼叫close.尤其是，如果在execute调用期间或在它与close调用之间引发了某些异常，则close可能不会被调用，从而导致泄漏.)

I am running a code for long hours as part of a stress test on an Oracle db and using java version "1.4.2". In a nutshell, what I am doing is :

while(true)
{
    Allocating some memory as a blob
    byte[] data = new byte[1000];
    stmt = fConnection.prepareStatement(query); // [compiling an insert query which uses the above blob]
    stmt.execute();  // I insert this blob-row in the database. 
stmt.close();

}

Now I want to run this test for 8-10 hrs. However apparently after inserting about 15million records I hit the java.lang.OutOfMemoryError

I am running this with -Xms512m -Xmx2g . I tried using higher values but I dont seem to have that much hardware neither do I think it is req:

    java -Xms512m -Xmx4g -jar XX.jar
    Invalid maximum heap size: -Xmx4g
    The specified size exceeds the maximum representable size.
    Could not create the Java virtual machine.
    java -Xms2g -Xmx3g -jar XX.jar
    Error occurred during initialization of VM
    Could not reserve enough space for object heap
    Could not create the Java virtual machine.

I run this as a multithreaded program. So arnd 10 threads are doing the inserts.

Is there any way I can get around this program in possibly no-hack manner. I mean what if I decide to run this for 15-20 hrs instead of 8-10 hrs.

EDIT: added stmt.close since I am using that in my code already. some changes based on comments

Thanks

P.S: sorry cant post the code bec of NDA

解决方案

Basically, I think you are barking up the wrong tree:

• The JVM / GC will manage to deallocate unreachable objects, no matter how fast you allocate them. If you are running the classic non-concurrent GC, then JVM will simply stop doing other things until the GC has deallocated memory. If you configured your JVM to use a concurrent GC, it will try to run the GC and normal worker threads at the same time ... and revert to "stop everything and collect" behaviour if it cannot keep up.

• If you are running out of memory, it is because something in your application (or the libraries / drivers it is using) is leaking memory. In other words, something is causing objects to remain reachable, even though your application doesn't need them any more.

As comments have pointed out, you need to address this problem methodically using a memory profiler / heap dump. Randomly changing things or blaming it on the GC is highly unlikely to fix the problem.

(When you say "... I did use stmt.close() all the time", I assume that this means that your code looks something like this:

    PreparedStatement stmt = ... 
    try {
        stmt.execute();
        // ...
    } finally {
        stmt.close();
    }

If you don't put the close call in a finally then it is possible that you are NOT calling close every time. In particular, if some exception gets thrown during the execute call or between it and the close call, then it is possible that close will not get called ... and that will result in a leak.)

这篇关于我的代码中的内存不足异常的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！