如何使用ImageIO插件解决OutOfMemoryError问题? [英] How to resolve OutOfMemoryError with ImageIO plugins as the cause?

问题描述

在工作中，我们有一些运行多个Web应用程序的tomcat服务器，其中约有一半必须进行一些图像处理.

At work we have some tomcat servers running several webapps, about half of which have to do some image processing.

在进行图像处理之前，这些Web应用程序会执行ImageIO.scanForPlugins()将适当的图像读取器和写入器存储到内存中.在需要处理图像的任何时候都运行此扫描之前，我们现在仅在初始化Webapp时运行扫描(由于我们在运行后未添加任何jar，为什么要多次运行扫描?)

Before doing their image processing, these webapps do a ImageIO.scanForPlugins() to get the appropriate image readers and writers into memory. While before this was just run anytime an image needed to be processed, we now run the scan only when the webapps are initialized (since we don't add any jars after running, why run the scan more than once?)

几天后，由于OutOfMemoryError，tomcat实例崩溃了.幸运的是，我们设置了HeapDumpOnOutOfMemoryError选项，因此我查看了堆转储.在转储中，我发现javax.imageio.spi.PartialOrderIterator的实例占用了97％的内存.它的大部分空间都由支持java.util.LinkedList占用，该元素有1800万个元素.链接列表由javax.imageio.spi.DigraphNode组成，其中包含ImageIO.scanForPlugins()加载的图像读取器和写入器.

Some days later, the tomcat instance crashed due to an OutOfMemoryError. Luckily we had the HeapDumpOnOutOfMemoryError option set, so I looked at the heap dump. In the dump I found that 97% of memory was taken by an instance of a javax.imageio.spi.PartialOrderIterator. Most of that space was taken up by it's backing java.util.LinkedList, which had 18 million elements. The linked list is made up of javax.imageio.spi.DigraphNode, which contains the image readers and writers loaded by ImageIO.scanForPlugins().

啊哈"，我想，我们必须在某处循环运行扫描，而我们只是一遍又一遍地添加相同的元素".但是，我认为我应该仔细检查这个假设，因此我编写了以下测试类:

"Aha", I thought, "we must be running the scan in a loop somewhere and we're just adding the same elements over and over again". But, I figured I should double check this assumption, so I wrote the following test class:

import javax.imageio.ImageIO;

public class ImageIOTesting {

public static void main(String[] args) {

    for (int i = 0; i < 100000; i++) {
        ImageIO.scanForPlugins();
        if (i % 1000 == 0) {
            System.out.println(Runtime.getRuntime().totalMemory() / 1024);
        }
    }
}
}

但是，当我在服务器环境上运行此类时，使用的内存量永远不会改变！

However, when I run this class on the server environment, the amount of memory in use never changes!

快速浏览一下javax.imageio软件包的源代码可以发现，该扫描检查是否已注册了服务提供者，如果已注册，则在注册新提供者之前先注销旧提供者.所以现在的问题是:为什么我会有这个庞大的服务提供商链接列表?为什么将它们存储为有向图?更重要的是，如何防止这种情况发生?

A quick dig through the source of the javax.imageio packages shows that the scan checks to see if a service provider is already registered, and if so it deregisters the old provider before registering the new one. So now the question is: Why do I have this giant linked list of service providers? Why are they stored as a directed graph? And more importantly, how do I prevent this from happening?

推荐答案

对一个老问题的最新答案，但无论如何:

Late answer to an old question, but anyway:

由于ImageIO插件注册表(IIORegistry)是"VM全局"，因此默认情况下，它不适用于servlet上下文.如果您像OP那样从WEB-INF/lib或classes文件夹加载插件，这一点尤其明显.

Because the ImageIO plugin registry (the IIORegistry) is "VM global", it doesn't by default work well with servlet contexts. This is especially evident if you load plugins from the WEB-INF/lib or classes folder, as the OP seems to do.

Servlet上下文动态加载和卸载类(每个上下文使用一个新的类加载器).如果重新启动应用程序，则默认情况下，旧类将永久保留在内存中(因为下次调用scanForPlugins时，它将扫描/加载类是另一个ClassLoader，因此它们将是注册表中的新实例.测试代码循环中，始终使用相同的ClassLoader，因此实例被替换，真正的问题永不显现.

Servlet contexts dynamically loads and unloads classes (using a new class loader per context). If you restart your application, old classes will by default remain in memory forever (because the next time scanForPlugins is called, it's another ClassLoader that scans/loads classes, and thus they will be new instances in the registry. In the test code loop, the same ClassLoader is used all the time, thus instances are replaced, and the real problem never manifests).

要变通解决此资源泄漏，我建议使用ContextListener以确保明确删除了这些本地上下文"插件.这是一个示例 IIOProviderContextListener 我已经在两个项目中使用过，这些项目实现了ImageIO插件的动态加载和卸载.

To work around this resource leak, I recommend using a ContextListener to make sure these "context local" plugins are explicitly removed. Here's an example IIOProviderContextListener I've used in a couple of projects, that implements dynamic loading and unloading of ImageIO plugins.

这篇关于如何使用ImageIO插件解决OutOfMemoryError问题?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！