内存不足错误，java堆空间 [英] out of memory error, java heap space

问题描述

我尝试读取超过400万行且大小超过400 MB的日志文件，但我得到内存不足错误：java堆空间。这是我的代码：

I try to read log file with more than 4 million lines and size more than 400 MB, but I get Out of Memory Error : java heap space. This is my code :

File file = new File("C:\\file.log");
        FileReader fileReader = new FileReader(file);
        BufferedReader bufferedReader = new BufferedReader(fileReader);
        StringBuilder stringBuffer = new StringBuilder();
        String line;
        while ((line = bufferedReader.readLine()) != null) {
            stringBuffer.append(line);
        }

我试图将堆内存增加到1GB，但仍然可以获得该消息。可能的原因是什么？

I tried to increase heap memory to 1GB, but still get that message. What would be the possible cause?

推荐答案

好的，你已经应该有一个线索，阅读你得到的评论。

Ok, you already should have a clue, reading the comments you got.

问题解释：

您的日志文件大小为400MB。请注意，这是以字节为单位。现在，您逐行读取 line = bufferedReader.readLine（），从而将一些字节转换为字符串。

Your log file has a size of 400MB. Note, that this is measured in bytes. Now you are reading it line by line with line = bufferedReader.readLine() thus converting some bytes to a string.

A String 实例在Java内部持有 char [] 。但Java中的 char 需要2个字节！所以你需要至少800MB的堆空间来存储所有的字符。由于您还要分配其他几个对象，并且JVM本身需要一些内存，因此很可能1 GB是不够的。

A String instance in Java internally holds a char[]. But a char in Java takes 2 bytes! So you need at least 800MB of heap space just for storing all the characters. As you are also allocating several other objects, and the JVM itself needs some memory, it is very probable that 1 GB is not enough.

此外， StringBuffer （顺便说一下：更好地使用 StringBuilder ）在内部再次使用 char [] ，在需要时自动扩展（长度）。这种扩展是通过加倍长度来完成的。所以对于一个400MB的文件，它有一个 char [] ，长度为512M。仍然提醒：一个字符需要2个字节。

Additionally, the StringBuffer (by the way: better use StringBuilder for that) internally uses again a char[], which is expanded (in length) automatically when needed. This expansion is done by doubling the length. So for a 400MB file it has a char[] with a length of 512M. Still remind: A char takes 2 bytes.

那么解决方案是什么？简单地说：不要将整个文件读入内存！

So what is the solution? Simply put: Do not read the entire file into memory!

请改为：

class LogAnalyzer {
    private final File logFile;

    LogAnalyzer(File logFile) {
        this.logFile = logFile;
    }

    void analyze() throws IOException {
        try(FileReader fileReader = new FileReader(logFile)) {
            try(BufferedReader bufferedReader = new BufferedReader(fileReader)) {
                String line;
                while ((line = bufferedReader.readLine()) != null) {
                    analyzeLine(line);
                }
            }
        }
    }

    private void analyzeLine(String line) {
        // do whatever you need here
    }
}

如果你需要保留一些行，你应该将它们存储在某些行中LogAnalyzer的实例字段，和/或让此类的行为类似于状态机。

If you need to keep some lines, you should store them in some instance fields of the LogAnalyzer, and/or have this class behave like a state machine.

这篇关于内存不足错误，java堆空间的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！