使用java.nio.MappedByteBuffer时防止OutOfMemory [英] Prevent OutOfMemory when using java.nio.MappedByteBuffer

问题描述

考虑应用程序，它创建5-6个线程，循环中的每个线程为5mb页面大小分配MappedByteBuffer。

Consider application, which create 5-6 threads, each thread in cycle allocate MappedByteBuffer for 5mb page size.

MappedByteBuffer b = ch.map(FileChannel.MapMode.READ_ONLY, r, 1024*1024*5);

当应用程序使用大文件时，迟早会抛出oom

Sooner or later, when application works with big files, oom is thrown

java.io.IOException: Map failed  at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:758)
Caused by: java.lang.OutOfMemoryError: Map failed
        at sun.nio.ch.FileChannelImpl.map0(Native Method)
        at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:755)

根据规范，MappedBuffer应该在GC本身时立即处置直接内存。看起来问题是，MappedBuffer-s的GC编辑太晚，后来直接内存完成。

According to specification, MappedBuffer should dispose direct memory as soon as it is GC itself. Looks like the problem is, that MappedBuffer-s are GC-ed too late, later then direct memory finished.

如何避免这种情况？可能会说MappedBuffer隐式处理或使用某种MappedBuffer池

How to avoid this situation ? Probably say MappedBuffer to dispose implicitly or use some kind of pool of MappedBuffer

推荐答案

您可以避免通过清理触发GC直接映射的字节缓冲区。

You can avoid having to trigger a GC by cleaning up the mapped byte buffers directly.

public static void clean(ByteBuffer bb) {
    if(bb == null) return;
    Cleaner cleaner = ((DirectBuffer) bb).cleaner();
    if(cleaner != null) cleaner.clean();
}

如果您在丢弃之前调用此方法，则不会耗尽虚拟内存。

Provided you call this before discarding, you won't run out of virtual memory.

也许您可以减少创建较大的ByteBuffers（除非您有大量文件）创建MappedByteBuffer不是免费的（大约需要50微秒）一些机器）

Perhaps you can look at creating larger ByteBuffers less often (unless you have a large number of files) Creating a MappedByteBuffer is not free (takes about 50 micro-seconds on some machines)

这篇关于使用java.nio.MappedByteBuffer时防止OutOfMemory的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！