如何处理 Java 中的 OutOfMemoryError? [英] How to handle OutOfMemoryError in Java?

问题描述

我必须序列化大约一百万个项目，并且在运行我的代码时出现以下异常:

I have to serialize around a million items and I get the following exception when I run my code:

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOfRange(Unknown Source)
    at java.lang.String.<init>(Unknown Source)
    at java.io.BufferedReader.readLine(Unknown Source)
    at java.io.BufferedReader.readLine(Unknown Source)
    at org.girs.TopicParser.dump(TopicParser.java:23)
    at org.girs.TopicParser.main(TopicParser.java:59)

我该如何处理?

推荐答案

我知道官方的 Java 回答是哦，不！记忆力不足！我屈服了！".对于在不允许将内存不足视为致命错误的环境(例如，编写操作系统，或为不受保护的操作系统编写应用程序)的环境中进行编程的任何人来说，这都是相当令人沮丧的.

I know that the official Java answer is "Oh noes! Out of memories! I give in!". This is all rather frustrating for anyone who has programmed in environments where running out of memory is not allowed to be a fatal error (for example, writing an OS, or writing apps for non-protected OSes).

放弃的意愿是必要的——你无法控制Java内存分配的方方面面，所以你不能保证你的程序在低内存情况下会成功.但这并不意味着你必须不战而降.

The willingness to surrender is necessary - you can't control every aspect of Java memory allocation, so you can't guarantee that your program will succeed in low-memory conditions. But that doesn't mean you must go down without a fight.

不过，在战斗之前，您可以寻找避免这种需要的方法.也许您可以避免 Java 序列化，而是定义您自己的数据格式，该格式不需要大量内存分配来创建.序列化会分配大量内存，因为它保留了之前见过的对象的记录，因此如果它们再次出现，它可以通过编号引用它们而不是再次输出它们(这可能导致无限循环).但那是因为它需要是通用的:根据您的数据结构，您可能能够定义一些文本/二进制/XML/任何可以写入流的表示，几乎不需要存储额外的状态.或者，您可以安排将您需要的任何额外状态一直存储在对象中，而不是在序列化时创建.

Before fighting, though, you could look for ways to avoid the need. Perhaps you can avoid Java serialization, and instead define your own data format which does not require significant memory allocation to create. Serialization allocates a lot of memory because it keeps a record of objects it has seen before, so that if they occur again it can reference them by number instead of outputting them again (which could lead to an infinite loop). But that's because it needs to be general-purpose: depending on your data structure, you might be able to define some text/binary/XML/whatever representation which can just be written to a stream with very little need to store extra state. Or you might be able to arrange that any extra state you need is stored in the objects all along, not created at serialization time.

如果您的应用程序执行一个使用大量内存的操作，但大部分使用的内存要少得多，特别是如果该操作是用户启动的，并且您找不到使用更少内存或提供更多内存的方法，那么可能值得捕获 OutOfMemory.您可以通过向用户报告问题太大来恢复，并邀请他们修剪并重试.如果他们只花了一个小时来解决他们的问题，你不想只想退出程序并失去一切——你想给他们一个机会做点什么.只要错误在堆栈中被捕获，在错误被捕获时多余的内存将不被引用，至少给 VM 一个恢复的机会.确保在常规事件处理代码下方捕获错误(在常规事件处理中捕获 OutOfMemory 可能导致繁忙循环，因为您尝试向用户显示对话框，但仍然没有内存，并且您捕获另一个错误).仅在您确定为内存占用的操作周围捕获它，以便您无法处理的 OutOfMemoryErrors 来自内存占用以外的代码，不会被捕获.

If your application does one operation which uses a lot of memory, but mostly uses much less, and especially if that operation is user-initiated, and if you can't find a way to use less memory or make more memory available, then it might be worth catching OutOfMemory. You could recover by reporting to the user that the problem is too big, and inviting them to trim it down and try again. If they've just spend an hour setting up their problem, you do not want to just bail out of the program and lose everything - you want to give them a chance to do something about it. As long as the Error is caught way up the stack, the excess memory will be unreferenced by the time the Error is caught, giving the VM at least a chance to recover. Make sure you catch the error below your regular event-handling code (catching OutOfMemory in regular event handling can result in busy loops, because you try to display a dialog to the user, you're still out of memory, and you catch another Error). Catch it only around the operation which you've identified as the memory-hog, so that OutOfMemoryErrors you can't handle, that come from code other than the memory hog, are not caught.

即使在非交互式应用程序中，放弃失败的操作也可能是有意义的，但程序本身可以继续运行，处理更多数据.这就是为什么 Web 服务器管理多个进程的原因，如果一个页面请求因内存不足而失败，服务器本身不会崩溃.正如我在顶部所说，单进程 Java 应用程序无法做出任何此类保证，但它们至少可以比默认设置更健壮一些.

Even in a non-interactive app, it might make sense to abandon the failed operation, but for the program itself to carry on running, processing further data. This is why web servers manage multiple processes such that if one page request fails for lack of memory, the server itself doesn't fall over. As I said at the top, single-process Java apps can't make any such guarantees, but they can at least be made a bit more robust than the default.

也就是说，您的特定示例(序列化)可能不适合这种方法.特别是，当用户被告知存在问题时，他们可能想做的第一件事是保存他们的工作:但如果序列化失败，则可能无法保存.这不是你想要的，所以你可能需要做一些实验和/或计算，并手动限制你的程序允许多少百万个项目(基于它运行的内存量)，之前它尝试序列化的点.

That said, your particular example (serialization) may not be a good candidate for this approach. In particular, the first thing the user might want to do on being told there's a problem is save their work: but if it's serialization which is failing, it may be impossible to save. That's not what you want, so you might have to do some experiments and/or calculations, and manually restrict how many million items your program permits (based on how much memory it is running with), before the point where it tries to serialize.

这比试图捕捉错误并继续执行更稳健，但不幸的是很难计算出确切的界限，因此您可能不得不谨慎行事.

This is more robust than trying to catch the Error and continue, but unfortunately it's difficult to work out the exact bound, so you would probably have to err on the side of caution.

如果错误是在反序列化过程中发生的，那么您的理由要牢固得多:如果您可以避免的话，未能加载文件不应该是应用程序中的致命错误.捕获错误可能更合适.

If the error is occurring during deserialization then you're on much firmer ground: failing to load a file should not be a fatal error in an application if you can possibly avoid it. Catching the Error is more likely to be appropriate.

无论你做什么来处理资源短缺(包括让错误关闭应用程序)，如果你关心后果，那么彻底测试它真的很重要.困难在于，您永远不知道代码中的哪一点会出现问题，因此通常有大量的程序状态需要测试.

Whatever you do to handle lack of resources (including letting the Error take down the app), if you care about the consequences then it's really important to test it thoroughly. The difficulty is that you never know exactly what point in your code the problem will occur, so there is usually a very large number of program states which need to be tested.

这篇关于如何处理 Java 中的 OutOfMemoryError?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！