与堆使用情况相比，Java应用程序的系统内存使用量巨大 [英] Huge system memory usage of java applications compared to heap usage

问题描述

我有一个像java框架的微服务。许多java进程在一个盒子上运行（ubuntu 14.04.4 LTS）。 java进程使用很多系统内存，因此交换空间被大量使用。 jstat gc报告不解释系统内存使用情况。所有java进程都使用参数运行
$ b $ pre $ -XX：MinHeapFreeRatio = 20 -XX：MaxHeapFreeRatio = 40 -XX：GCTimeRatio = 4 - XX：AdaptiveSizePolicyWeight = 90


强制JVM将内存传回给系统。没有参数，问题依然存在。一些java组件使用nashorn引擎来编写一些功能。

有人可以在这里解释行为吗？

是否有任何限制巨大系统内存使用的jvm patameters？

如何命令操作系统对jvm的内存分配有更多限制？

有些数据：

组件A（与nashorn）

top：

  PID用户PR NI VIRT RES SHR S％CPU％MEM TIME + COMMAND 
 2400 xxxxxx 20 0 13.933g 807496 7332 S 0.0 2.5 4180：15 java 
  

jstat -gc 2400：

• 容量：约。 180 MB


• 用法：约。 165 MB


• 系统资源：约。 800 MB
  / b>
  
  

  为什么组件使用的GC区域内存是4倍多？

  
  
  

  组件B（不包括nashorn）
  
  

  top：
  
  

    PID USER PR PR NI VIRT RES SHR S％CPU％MEM TIME + COMMAND 
   19476 xxxx 20 0 13.465g 120436 7836 S 7.0 0.4 22：40.76 java 
    

  jstat -gc 19476：
  
  

    S0C S1C S0U S1U EC EU OC OU MC MU CCSC CCSU YGC YGCT FGC FGCT GCT 
   512.0 512.0 0.0 0.0 41472.0 25408.7 343040.0 7164.5 17664.0 17183.1 2048.0 1919.4 3650 10.806 939 16.788 27.594 
    pre> 
   
   
   
   
  容量：约。 403 MB 
   
   
  用法：约。 52 MB  
   
  系统资源：120 MB  
   
   
   
   
   GC区域容量大于实际系统内存使用情况系统内存使用量仍然是GC区域的两倍。 IMO此组件行为正常，因为库等也部分映射到内存中。
   
   
   
   组件C（不含nashorn） 
   
   
   
   top： 
   
   
    PID用户PR NI VIRT RES SHR S％CPU％MEM TIME + COMMAND 
   2272 xxxxxx 20 0 13.382g 922944 11108 S 0.7 2.8 40033：41 java 
    
   jstat -gc 2272：
   
   $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ s $ 1024.0 868.0 0.0 36352.0 23866.1 76800.0 56580.2 68864.0 64571.1 8448.0 7460.6 31974159 199295.501 844692 134644.040 333939.541 
    

  
  
  

  
  <李>容量：约。 190 MB
  
  • 用法：约。 152 MB
    
  • 系统资源：920 MB
    
  
  
  

  为什么组件使用次数多于6次GC区域内存？

  解决方案

  
  

  有人可以在这里解释行为吗？ $ b

  对于内存使用情况没有单一的解释，有很多促成因素。
  
  

  
  
  • 使用 NMT 获取概述多少内存由JVM的各个内部部分分配
    
  • 使用 pmap -x > 来标识内存
    
  • 使用堆转储来查找直接内存缓冲区（它们仅显示为 [anon] 在pmap中映射）或使用 Yourkit 进行内存检查以确定分配的数量通过直接缓冲区。在运行时，您可以使用 BufferPoolMXBean 以跟踪直接缓冲区的使用情况。除此之外，您必须考虑到每个JVM都带有一些基线内存消耗和需要呼吸垃圾收集器的空间。在共享JVM中运行多个服务可以分摊这些基准成本。
    
    
    

    由于虚拟内存系统的复杂性，您还需要了解在使用，提交，保留和驻留内存之间的差异。
    
    

    
    

    是否有任何限制系统内存使用量的jvm patameters？

    
    

    这取决于原因。

    
    
    

    对于托管堆，可以使其更快地将未使用的内存退回到操作系统，但有性能损失。

    
    

    I have a microservice like java framework. Many java processes run on a single box (ubuntu 14.04.4 LTS). The java processes use alot system memory, so the swap space is used heavily. The jstat gc reports do not explain the systems memory usage. All java processes run with the parameters

    -XX:MinHeapFreeRatio=20 -XX:MaxHeapFreeRatio=40 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90
    

    to force the JVM to give memory back to the system. Without the parameters the problem persists. Some java components use nashorn engine to script some functionality.

    Can someone explain the behavior here?

    Are there any jvm patameters that restrict the huge system memory usage?

    How to command the OS to be more restrictive with memory allocation for the jvm?

    Some data:

    Component A (with nashorn)

    top:

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
    2400 xxxxxx    20   0 13.933g 807496   7332 S   0.0  2.5   4180:15 java
    

    jstat -gc 2400:

    S0C    S1C    S0U    S1U      EC       EU        OC         OU       MC     MU    CCSC   CCSU   YGC     YGCT    FGC    FGCT     GCT
    512.0  512.0   0.0   400.0  19456.0  12751.6   62464.0    59862.3   89688.0 84866.6 10624.0 9440.4 2165265 15977.896 16816 1813.836 17791.732
    

    • capacity: ca. 180 MB
    • usage: ca. 165 MB
    • system resources: ca. 800 MB

    Why does the component uses more the 4 times the GC areas memory?

    Component B (without nashorn)

    top:

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
    19476  xxxx     20   0 13.465g 120436   7836 S   7.0  0.4  22:40.76 java
    

    jstat -gc 19476:

    S0C    S1C    S0U    S1U      EC       EU        OC         OU       MC     MU    CCSC   CCSU   YGC     YGCT    FGC    FGCT     GCT
    512.0  512.0   0.0    0.0   41472.0  25408.7   343040.0    7164.5   17664.0 17183.1 2048.0 1919.4   3650   10.806  939    16.788   27.594
    

    • capacity: ca. 403 MB
    • usage: ca. 52 MB
    • system resources: 120 MB

    Here the GC area capacities are bigger than the actual systems memory usage. Still the systems memory usage is twice the GC areas. IMO this component behaves normal because the libraries etc. are mapped partially into memory as well.

    Component C (without nashorn)

    top:

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
    2272 xxxxxx    20   0 13.382g 922944  11108 S   0.7  2.8  40033:41 java
    

    jstat -gc 2272:

    S0C    S1C    S0U    S1U      EC       EU        OC         OU       MC     MU    CCSC   CCSU   YGC     YGCT    FGC    FGCT     GCT
    1024.0 1024.0 868.0   0.0   36352.0  23866.1   76800.0    56580.2   68864.0 64571.1 8448.0 7460.6 31974159 199295.501 844692 134644.040 333939.541
    

    • capacity: ca. 190 MB
    • usage: ca. 152 MB
    • system resources: 920 MB

    Why does the component uses more the 6 times the GC areas memory?

    解决方案

    Can someone explain the behavior here?

    There is no single explanation for memory usage, there are many contributing factors.

    • Use NMT to get an overview of how much memory is allocated by various internal parts of the JVM
    • use pmap -x <pid> to identify memory-mapped files.
    • take a heap dump to look for direct memory buffers (they just show up as some of the [anon] mappings in pmap) or use Yourkit which has a memory inspection to identify the amount allocated by direct buffers. At runtime you can use BufferPoolMXBean to track direct buffer use.

    Beyond that, you have to take into account that each JVM comes with some baseline memory consumption and needed breathing room for the garbage collector. Running multiple services in a shared JVM can amortize those baseline costs.

    Due to complexities of the virtual memory system you also need to be aware of the difference between used, committed, reserved and resident memory.

    Are there any jvm patameters that restrict the huge system memory usage?

    That would depend on the cause.

    For the managed heap it's possible to make it yield unused memory back to the OS more swiftly, but comes with a performance penalty.

    这篇关于与堆使用情况相比，Java应用程序的系统内存使用量巨大的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！