设置-XX的影响：使用NIO直接缓冲区时+ DisableExplicitGC [英] Impact of setting -XX:+DisableExplicitGC when NIO direct buffers are used
问题描述
我们正在构建一个具有高性能SLA的Web应用程序，由于System.gc（）调用JVM会午餐，因此定期违反SLA。我们已经做了一些调试，并确定在所有情况下它都是调用System.gc（）的内部应用服务器代码。这在应用程序服务器启动或部署应用程序时发生了几次，这是我们不关心的。但是，当应用程序启动并通过内部应用程序服务器调用NIO类运行时，也会定期触发System.gc（）。以下是我们能够捕获此事件的堆栈跟踪：
3XMTHREADINFOWebContainer：25J9VMThread：0x0000000006FC5D00，j9thread_t：0x00007F60E41753E0 ，java / lang / Thread：0x000000060B735590，state：R，prio = 5
3XMJAVALTHREAD（java / lang / Thread getId：0xFE，isDaemon：true）
3XMTHREADINFO1（native thread ID：0x1039，native priority： 0x5，本地策略：UNKNOWN）
3XMTHREADINFO2（本地堆栈地址范围：0x00007F6067621000，至：0x00007F6067662000，大小：0x41000）
3XMCPUTIME CPU使用总数：80.222215853秒
3XMHEAPALLOC自上次分配以来分配的堆字节GC周期= 1594568（0x1854C8）
3XMTHREADINFO3 Java调用堆栈：
java / lang / System.gc处的4XESTACKTRACE（System.java:329）
java / nio处的4XESTACKTRACE / Bits.syncReserveMemory .java：721）
5XESTACKTRACE（输入锁定：java / nio / Bits @ 0x000000060000B690，条目计数：1 ）
java / nio / Bits.reserveMemory处的4XESTACKTRACE（Bits.java:766（编译后的代码））
java / nio / DirectByteBuffer处的4XESTACKTRACE< init>（DirectByteBuffer.java:123（Compiled Code ））
4XESTACKTRACE在java / nio / ByteBuffer.allocateDirect（ByteBuffer.java:306（Compiled Code））
4XESTACKTRACE在com / ibm / ws / buffermgmt / impl / WsByteBufferPoolManagerImpl.allocateBufferDirect（WsByteBufferPoolManagerImpl.java： 706（Compiled Code））
在com / ibm / ws / buffermgmt / impl / WsByteBufferPoolManagerImpl.allocateCommon中的4XESTACKTRACE（WsByteBufferPoolManagerImpl.java:612（编译代码））
在com / ibm / ws / buffermgmt / impl / WsByteBufferPoolManagerImpl.allocateDirect（WsByteBufferPoolManagerImpl.java:527（编译代码））
4XESTACKTRACE at com / ibm / io / async / ResultHandler.runEventProcessingLoop（ResultHandler.java:507（编译代码））
4XESTACKTRACE at COM / IBM / IO /异步/ ResultH andler $ 2.run（ResultHandler.java:905（Compiled Code））
在com / ibm / ws / util / ThreadPool中的4XESTACKTRACE $ Worker.run（ThreadPool.java:1864（编译的代码））
3XMTHREADINFO3本地调用堆栈：
4XENATIVESTACK（0x00007F61083DD122 [libj9prt26.so + 0x13122]）
4XENATIVESTACK（0x00007F61083EA79F [libj9prt26.so + 0x2079f]）
....
有人知道如果我们通过启用-XX关闭对System.gc（）的调用会产生什么样的影响：+ DisableExplicitGC（或实际上在我们的例子中，通过设置-Xdisableexplicitgc，因为我们在IBM JRE上运行Websphere，它也是这样做的）？我们当然不想创建内存泄漏。我无法直接找到为什么NIO中的System.gc（）调用实际上是必需的，并且没有代码注释特别指出它在JDK代码中出现的位置： http：// hg.openjdk.java.net/jdk8u/jdk8u-dev/jdk/file/4a1e42601d61/src/share/classes/java/nio/Bits.java
如果由于使用NIO而完全禁用System.gc（）是一个糟糕的主意，那么至少我们可以做些什么来减少它被调用的频率？看起来，我们可以设置-XX：MaxDirectMemorySize，但是这看起来好像只会设置分配内存量的上限，并且可能会产生负面影响。
这意味着直接缓冲区分配的内存在收集之前可能会积累很长时间。从长远来看，这不是真正的泄漏，但它会增加峰值内存使用量。
据我所知， system.gc（）调用在 reserveMemory 限制为击中。保留请求的数量 ByteBuffer.allocateDirect 后，将会调用 Unsafe.allocateMemory ，这可能会进行自己的GC调用，应该不会受到 DisableExplicitGC 的影响，如果其尝试mmap失败。
至少我们可以做些什么来减少它被调用的频率？
只有当 MaxDirectMemorySize 限制被击中。如果您可以调整您的GC或应用程序代码，以便它符合下列选项之一：
- 它使用一组固定的缓冲区（ - >限制将永远不会超过）
- 缓冲区是早期收集的（短暂缓冲区 - >在一个年轻的GC中死亡）
- 定期收集老一代直接缓冲区空间用完之前
- 使用堆缓冲而不是直接缓冲
然后 System.gc（）不需要调用。
在热点上还有一个 ExplicitGCInvokesConcurrent 选项。也许IBM的VM有类似的东西。
We are building a web application with aggressive performance SLAs which are periodically being violated due to the JVM going out to lunch due to System.gc() calls. We've done some debugging, and determined that in all cases it's the internal app server code which is invoking System.gc(). This occurs a few times while the app server is booting or the application is deployed, which we aren't concerned about. However, System.gc() is also periodically be triggered when the app is up and running via internal app server calls the NIO classes. Here's a stack trace we were able to capture of this event:
3XMTHREADINFO "WebContainer : 25" J9VMThread:0x0000000006FC5D00, j9thread_t:0x00007F60E41753E0, java/lang/Thread:0x000000060B735590, state:R, prio=5
3XMJAVALTHREAD (java/lang/Thread getId:0xFE, isDaemon:true)
3XMTHREADINFO1 (native thread ID:0x1039, native priority:0x5, native policy:UNKNOWN)
3XMTHREADINFO2 (native stack address range from:0x00007F6067621000, to:0x00007F6067662000, size:0x41000)
3XMCPUTIME CPU usage total: 80.222215853 secs
3XMHEAPALLOC Heap bytes allocated since last GC cycle=1594568 (0x1854C8)
3XMTHREADINFO3 Java callstack:
4XESTACKTRACE at java/lang/System.gc(System.java:329)
4XESTACKTRACE at java/nio/Bits.syncReserveMemory(Bits.java:721)
5XESTACKTRACE (entered lock: java/nio/Bits@0x000000060000B690, entry count: 1)
4XESTACKTRACE at java/nio/Bits.reserveMemory(Bits.java:766(Compiled Code))
4XESTACKTRACE at java/nio/DirectByteBuffer.<init>(DirectByteBuffer.java:123(Compiled Code))
4XESTACKTRACE at java/nio/ByteBuffer.allocateDirect(ByteBuffer.java:306(Compiled Code))
4XESTACKTRACE at com/ibm/ws/buffermgmt/impl/WsByteBufferPoolManagerImpl.allocateBufferDirect(WsByteBufferPoolManagerImpl.java:706(Compiled Code))
4XESTACKTRACE at com/ibm/ws/buffermgmt/impl/WsByteBufferPoolManagerImpl.allocateCommon(WsByteBufferPoolManagerImpl.java:612(Compiled Code))
4XESTACKTRACE at com/ibm/ws/buffermgmt/impl/WsByteBufferPoolManagerImpl.allocateDirect(WsByteBufferPoolManagerImpl.java:527(Compiled Code))
4XESTACKTRACE at com/ibm/io/async/ResultHandler.runEventProcessingLoop(ResultHandler.java:507(Compiled Code))
4XESTACKTRACE at com/ibm/io/async/ResultHandler$2.run(ResultHandler.java:905(Compiled Code))
4XESTACKTRACE at com/ibm/ws/util/ThreadPool$Worker.run(ThreadPool.java:1864(Compiled Code))
3XMTHREADINFO3 Native callstack:
4XENATIVESTACK (0x00007F61083DD122 [libj9prt26.so+0x13122])
4XENATIVESTACK (0x00007F61083EA79F [libj9prt26.so+0x2079f])
....
Is anyone aware of what the impact would be if we shut down the calls to System.gc() by enabling -XX:+DisableExplicitGC (or actually in our case by setting -Xdisableexplicitgc since we're running Websphere on the IBM JRE, which does the same thing)? We certainly don't want to create a memory leak. I haven't been able to find a direct reference as to why the System.gc() calls in NIO are actually necessary, and there isn't a code comment specifically addressing it where it occurs in JDK code, either: http://hg.openjdk.java.net/jdk8u/jdk8u-dev/jdk/file/4a1e42601d61/src/share/classes/java/nio/Bits.java
If it's a bad idea to completely disable System.gc() due to the use of NIO, is there at least something we can do to reduce the frequency at which it's called? It appears that we can set -XX:MaxDirectMemorySize, but this appears as if it would only set up upper bound on the amount of allocated memory, and would just as likely have an adverse affect.
Disabling explicit GCs does not prevent buffers and thus the native memory they hold onto from being collected. But it may delay collections for a long time.
Which means memory allocated by direct buffers may accumulate for a long time before it is collected. In the long run that's not really a leak, but it will increase peak memory usage.
As I understand it that System.gc() call is there to free buffers when the reserveMemory limit is hit. After reserving the requested amount ByteBuffer.allocateDirect will call to Unsafe.allocateMemory which might do its own GC call, which should not be affected by DisableExplicitGC, if its attempt to mmap fails.
is there at least something we can do to reduce the frequency at which it's called?
It is only called when the MaxDirectMemorySize limitation is hit. If you can tune your GC or application code so that it meets one of the following options:
- it uses a fixed set of buffers (-> limitation will never be exceeded)
- buffers are collected early (shortlived buffers -> die in a young GC)
- the old generation gets collected regularly before direct buffer space runs out
- uses heap buffers instead of direct buffers
then System.gc() calls won't be necessary.
On hotspot there also exists a ExplicitGCInvokesConcurrent option. Maybe IBM's VM has something similar.
这篇关于设置-XX的影响：使用NIO直接缓冲区时+ DisableExplicitGC的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
