ZGC最大堆大小超过物理内存 [英] ZGC max heap size exceed physical memory

问题描述

1. JVM选项是

-server -Xmx100g -Xms100g -XX:MaxMetaspaceSize=1G -Xss512k 
-XX:+UnlockExperimentalVMOptions -XX:+UseZGC -XX:MaxGCPauseMillis=30 
-XX:ConcGCThreads=4 -XX:ParallelGCThreads=12 
-XX:+DisableExplicitGC -XX:LargePageSizeInBytes=128m

1. RAM为256G

             total       used       free     shared    buffers     cached
Mem:           251        250          1        100          0        138
-/+ buffers/cache:        112        139
Swap:            7          0          7

1. 最上面的命令显示进程的RES为303G

   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                           
240731 xxx       20   0 17.0t 302g 297g S  6.6 119.9 256:35.43 java

1. jvm个人资料显示如下

./jhsdb jmap --heap --pid 240731
Attaching to process ID 240731, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 11.0.2+9

using thread-local object allocation.
ZGC with 12 thread(s)

Heap Configuration:
   MinHeapFreeRatio         = 40
   MaxHeapFreeRatio         = 70
   MaxHeapSize              = 107374182400 (102400.0MB)
   NewSize                  = 1363144 (1.2999954223632812MB)
   MaxNewSize               = 17592186044415 MB
   OldSize                  = 5452592 (5.1999969482421875MB)
   NewRatio                 = 2
   SurvivorRatio            = 8
   MetaspaceSize            = 21807104 (20.796875MB)
   CompressedClassSpaceSize = 1073741824 (1024.0MB)
   MaxMetaspaceSize         = 1073741824 (1024.0MB)
   G1HeapRegionSize         = 0 (0.0MB)

Heap Usage:
 ZHeap          used 82988M, capacity 1024M, max capacity 27112449862M

是否有其他VM选项可提供最大容量?ZGC堆如何工作?

Any other VM options for the max capatity ? How the ZGC heap works ?

推荐答案

由于不时提出此问题，并且我已经厌倦了对此问题的解释，因此，我尝试将其放在此处，以便人们可以简单地进行搜索，获取他们的答案，并再次感到高兴(即使在很短的时间内，仍然值得！).

Since this question was asked from time to time, and I am tired of explaining it already, let me try to put it here so people can simply search, get their answer, and be happy again (even for a very short period of time, it's still worth it!).

RSS过度报告归因于ZGC用于操纵内存页面的技术，即多重映射.而且由于ZGC本质上是Azing Systems的Zing C4收集器的另一种实现，因此Zing也存在相同的过度报告" RSS问题.

RSS over-reporting by ZGC is due to the technique that ZGC uses to manipulate the memory pages, namely, multi-mapping. and since ZGC is essentially another implementation of Zing C4 collector (by Azul Systems), Zing also shares the same "over-reporting" RSS issue.

查看以下代码:

http://hg.openjdk.java.net/zgc/zgc/file/59c07aef65ac/src/hotspot/os_cpu/linux_x86/zPhysicalMemoryBacking_linux_x86.cpp#l160

void ZPhysicalMemoryBacking::map(ZPhysicalMemory pmem, uintptr_t offset) const {
  if (ZUnmapBadViews) {
    // Only map the good view, for debugging only
    map_view(pmem, ZAddress::good(offset), AlwaysPreTouch);
  } else {
    // Map all views
    map_view(pmem, ZAddress::marked0(offset), AlwaysPreTouch);
    map_view(pmem, ZAddress::marked1(offset), AlwaysPreTouch);
    map_view(pmem, ZAddress::remapped(offset), AlwaysPreTouch);
  }
}

这:

void ZPhysicalMemoryBacking::map_view(ZPhysicalMemory pmem, uintptr_t addr, bool pretouch) const {
  const size_t nsegments = pmem.nsegments();
  // Map segments
  for (size_t i = 0; i < nsegments; i++) {
    const ZPhysicalMemorySegment segment = pmem.segment(i);
    const size_t size = segment.size();
    const void* const res = mmap((void*)addr, size, PROT_READ|PROT_WRITE, MAP_FIXED|MAP_SHARED, _file.fd(), segment.start());
    if (res == MAP_FAILED) {
      ZErrno err;
      map_failed(err);
    }
    // Advise on use of transparent huge pages before touching it
    if (ZLargePages::is_transparent()) {
      advise_view(addr, size);
    }
    // NUMA interleave memory before touching it
    ZNUMA::memory_interleave(addr, size);
    if (pretouch) {
      pretouch_view(addr, size);
    }
    addr += size;
  }
}

映射所有视图:

• map_view(pmem，ZAddress :: marked0(offset)，AlwaysPreTouch);
• map_view(pmem，ZAddress :: marked1(offset)，AlwaysPreTouch);
• map_view(pmem，ZAddress :: remapped(offset)，AlwaysPreTouch);

这意味着对于相同的地址，ZGC会将其映射到3个不同的视图:marked0，marked1和重新映射.这3个视图反映在虚拟内存地址中.这意味着将3个不同的虚拟内存地址映射到相同的基础物理内存，因此，对于每个物理内存页面，都有3个虚拟页面映射到该虚拟内存地址.

This means that for the same address, ZGC will map it to 3 different views: marked0, marked1 and remapped. These 3 views are reflected int the virtual memory addresses. And this means, 3 different virtual memory addresses will be mapped to the same underlying physical memory, and thus for each physical memory page, there are 3 virtual pages mapped to it.

如果ZGC成长为一个世代GC(您拥有年轻一代和老一代，而不是像现在的ZGC那样只有一个世代)，那么我们可以预期这个数字也将增加到xmx堆大小的6倍

And if ZGC grows up to be a generational GC (where you have young generation and old generation, instead of one single generation as ZGC is right now), we can expect this number to be increased to 6x of xmx heap size too.

这就是为什么Azul Zing和Oracle ZGC都使用多重映射使人们在"top"命令处进入恐慌模式的原因.但是请注意，只有虚拟内存空间会被报告，因此，除非您的系统工具遇到这个令人困惑的部分，否则您没有理由致电911.

And this is why the multi-mapping used by both Azul Zing and Oracle ZGC put people to panic mode at "top" command. But please note that it is only the virtual memory space that gets reported, so unless your system tool runs into this confusing part, there is no reason for you to call 911.

这篇关于ZGC最大堆大小超过物理内存的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！