与堆使用相比，Java 应用程序的系统内存使用量巨大 [英] Huge system memory usage of java applications compared to heap usage

问题描述

我有一个像 Java 框架这样的微服务.许多 java 进程在一个机器上运行(ubuntu 14.04.4 LTS).Java 进程使用大量系统内存，因此大量使用交换空间.jstat gc 报告没有解释系统内存使用情况.所有java进程都带参数运行

I have a microservice like java framework. Many java processes run on a single box (ubuntu 14.04.4 LTS). The java processes use alot system memory, so the swap space is used heavily. The jstat gc reports do not explain the systems memory usage. All java processes run with the parameters

-XX:MinHeapFreeRatio=20 -XX:MaxHeapFreeRatio=40 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90

强制JVM将内存还给系统.如果没有参数，问题仍然存在.一些 Java 组件使用 nashorn 引擎来编写某些功能的脚本.

to force the JVM to give memory back to the system. Without the parameters the problem persists. Some java components use nashorn engine to script some functionality.

有人可以解释一下这里的行为吗?

Can someone explain the behavior here?

是否有限制巨大系统内存使用的jvm参数?

Are there any jvm patameters that restrict the huge system memory usage?

如何命令操作系统对 jvm 的内存分配进行更多限制?

How to command the OS to be more restrictive with memory allocation for the jvm?

一些数据:

组件 A(带鼻角)

顶部:

PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
2400 xxxxxx    20   0 13.933g 807496   7332 S   0.0  2.5   4180:15 java

jstat -gc 2400:

jstat -gc 2400:

S0C    S1C    S0U    S1U      EC       EU        OC         OU       MC     MU    CCSC   CCSU   YGC     YGCT    FGC    FGCT     GCT
512.0  512.0   0.0   400.0  19456.0  12751.6   62464.0    59862.3   89688.0 84866.6 10624.0 9440.4 2165265 15977.896 16816 1813.836 17791.732

• 容量:约.180 MB
• 用法:约.165 MB
• 系统资源:约.800 MB

为什么组件使用了 4 倍的 GC 区域内存?

Why does the component uses more the 4 times the GC areas memory?

组分 B(不含鼻角)

顶部:

PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
19476  xxxx     20   0 13.465g 120436   7836 S   7.0  0.4  22:40.76 java

jstat -gc 19476:

jstat -gc 19476:

S0C    S1C    S0U    S1U      EC       EU        OC         OU       MC     MU    CCSC   CCSU   YGC     YGCT    FGC    FGCT     GCT
512.0  512.0   0.0    0.0   41472.0  25408.7   343040.0    7164.5   17664.0 17183.1 2048.0 1919.4   3650   10.806  939    16.788   27.594

• 容量:约.403 MB
• 用法:约.52 MB
• 系统资源:120 MB

这里的 GC 区域容量大于实际系统内存使用量.系统内存使用量仍然是 GC 区域的两倍.IMO 这个组件表现正常，因为库等也部分映射到内存中.

Here the GC area capacities are bigger than the actual systems memory usage. Still the systems memory usage is twice the GC areas. IMO this component behaves normal because the libraries etc. are mapped partially into memory as well.

组件 C(不含鼻角)

顶部:

PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
2272 xxxxxx    20   0 13.382g 922944  11108 S   0.7  2.8  40033:41 java

jstat -gc 2272:

jstat -gc 2272:

S0C    S1C    S0U    S1U      EC       EU        OC         OU       MC     MU    CCSC   CCSU   YGC     YGCT    FGC    FGCT     GCT
1024.0 1024.0 868.0   0.0   36352.0  23866.1   76800.0    56580.2   68864.0 64571.1 8448.0 7460.6 31974159 199295.501 844692 134644.040 333939.541

• 容量:约.190 MB
• 用法:约.152 MB
• 系统资源:920 MB

为什么组件使用了 6 倍的 GC 区域内存?

Why does the component uses more the 6 times the GC areas memory?

推荐答案

有人可以解释一下这里的行为吗?

Can someone explain the behavior here?

内存使用没有单一的解释，有很多影响因素.

There is no single explanation for memory usage, there are many contributing factors.

• 使用 NMT 获得JVM 各个内部部分分配了多少内存的概览
• 使用 pmap -x 来识别内存映射文件.
• 进行堆转储以查找直接内存缓冲区(它们仅显示为 pmap 中的某些 [anon] 映射)或使用 Yourkit 有一个内存检查来识别直接缓冲区分配的数量.在运行时，您可以使用 BufferPoolMXBean跟踪直接缓冲区的使用.
• Use NMT to get an overview of how much memory is allocated by various internal parts of the JVM
• use pmap -x <pid> to identify memory-mapped files.
• take a heap dump to look for direct memory buffers (they just show up as some of the [anon] mappings in pmap) or use Yourkit which has a memory inspection to identify the amount allocated by direct buffers. At runtime you can use BufferPoolMXBean to track direct buffer use.

除此之外，您还必须考虑到每个 JVM 都有一些基准内存消耗，并且需要为垃圾收集器提供喘息空间.在共享 JVM 中运行多个服务可以分摊这些基准成本.

Beyond that, you have to take into account that each JVM comes with some baseline memory consumption and needed breathing room for the garbage collector. Running multiple services in a shared JVM can amortize those baseline costs.

由于虚拟内存系统的复杂性，您还需要了解已用内存、已提交内存、保留内存和常驻内存之间的区别.

Due to complexities of the virtual memory system you also need to be aware of the difference between used, committed, reserved and resident memory.

是否有限制巨大系统内存使用的jvm参数?

Are there any jvm patameters that restrict the huge system memory usage?

这取决于原因.

对于托管堆，可以使其更快地将未使用的内存返回给操作系统，但带有性能损失.

For the managed heap it's possible to make it yield unused memory back to the OS more swiftly, but comes with a performance penalty.

这篇关于与堆使用相比，Java 应用程序的系统内存使用量巨大的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！