OutOfMemoryError:Java堆空间.如何解决此递归方法中发生的错误? [英] OutOfMemoryError: Java Heap Space. How can I fix this error that happens in a recursive method?

问题描述

我有一个Java应用程序，可以解析目录及其子目录中的pdf文件，并使用文件中的信息创建数据库.

I have a Java application that parses pdf files in a directory and its subdirectories and creates a database using the information found in the files.

当我在大约900个文件上使用该程序时，一切都很好(它创建了一个具有多个表的SQLite数据库，其中有些包含15万行).

Everything was fine when I was using the program on around 900 files or so (which create a SQLite database with multiple tables, some of wihch contain 150k rows).

现在，我正在尝试在更大的数据集(大约2000个文件)上运行我的程序，有时我会收到"OutOfMemoryError:Java堆空间".我在jdev.conf文件中更改了以下行:

Now I'm trying to run my program on a larger set of data (around 2000 files) and at some point I get "OutOfMemoryError: Java Heap space". I changed the following line in my jdev.conf file:

AddVMOption  -XX:MaxPermSize=256M

到512M，我得到了相同的错误(不过我认为是稍后出现).我将再次将其更改为更大的体积，但问题是将要使用此程序的计算机更旧，因此没有足够的内存.通常，用户一次不会添加30个以上的文件，但是我想知道我应该将它们限制为多少个文件.理想情况下，无论要解析多少文件，我都希望程序不要引发错误.

to 512M and I got the same error (though later, I think). I'm going to change it to something bigger again, but the thing is the computers this program will be used on are much older and thus don't have as much memory. Normally, the users are not going to add more than 30 files at a time, but I want to know at how many files I'm supposed to limit them to. Ideally, I'd like my program not to throw an error regardless of how many files are to be parsed.

起初，我认为是导致错误的原因是我的SQLite查询，但在Google上阅读后，可能是一些递归函数.我将其隔离(至少我认为它是正确的)，以实现此功能:

At first, I thought it was my SQLite queries that were causing the error, but after reading up on Google, it's probably some recursive function. I isolated it (I think it's the correct one at least), to this function:

 public static void visitAllDirsAndFiles(File dir) {
      if(dir.isDirectory()) 
      {
        String[] children = dir.list();
        for (int i=0; i<children.length; i++) 
        {
          visitAllDirsAndFiles(new File(dir, children[i]));
        }
      }
      else
      {
        try
        {          
          BowlingFilesReader.readFile(dir);
        }
        catch(Exception exc)
        {
          exc.printStackTrace();
          System.out.println("Other Exception in file: " + dir);
        }
      }
  }

我认为问题可能在于它为每个后续目录递归调用了此函数，但我真的不确定这可能是问题所在.你怎么认为?如果可能的话，我该怎么做，这样我就不会再出现此错误了?如果您认为仅此部分是不可能导致问题的，我将尝试查找该程序的其他部分是否会导致此问题.

I think the problem might be that it recursively calls this function for each subsequent directory, but I'm really not sure that could be the problem. What do you think? If it might be, how can I make it so I don't get this error again? If you think it is impossible that this section alone causes the problem, I'll try to find which other part of the program can cause it.

我能看到的唯一另一件事是，在调用上述方法之前，我已连接到数据库，而在返回该方法之后，我断开了连接.这样做的原因是，如果我在每个文件之后都进行连接和断开连接，则我的程序将花费更长的时间来解析数据，因此我真的不想不必更改它.

The only other thing I can see causing that is that I connect to the database before calling the above method and I disconnect after it returns. The reason for that is that if I connect and disconnect after each file, my programs takes a lot longer to parse the data, so I'd really like not to have to change that.

推荐答案

如果问题的根源是递归，则将得到与堆栈有关的错误，而不是与堆有关的错误.似乎您在BowlingFilesReader ...

If the origin of the problem was recursion, you would get an error related to stack instead of heap. Seems that you have some kind of memory leak in BowlingFilesReader...

这篇关于OutOfMemoryError:Java堆空间.如何解决此递归方法中发生的错误?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！