OpenJDK JVM是否会将堆内存返回给Linux？ [英] Will OpenJDK JVM ever give heap memory back to Linux?

问题描述

我们有一个长期存在的服务器进程，很快就需要大量的RAM。我们看到，一旦JVM从操作系统获得内存，它的
永远不会将它返回给操作系统。我们如何要求JVM将堆内存返回给操作系统？

通常，这些问题的接受答案是使用
-XX：MaxHeapFreeRatio 和 -XX：MinHeapFreeRatio 。 （参见例如

显然，JVM不遵守 -XX：MaxHeapFreeRatio = 50 因为heapFreeRatio非常接近100％而且远不及50％。 。点击 执行GC 的量没有返回存储到OS

MemoryUsage.java：

  import java.util.ArrayList; 
 import java.util.List; 
 
 public class MemoryUsage {
 
 public static void main（String [] args）throws InterruptedException {
 System.out.println（在分配内存之前休眠） ; 
 Thread.sleep（10 * 1000）; 
 
 System.out.println（分配/增长内存）; 
列表< Long> list = new ArrayList<>（）; 
 //实验确定的因子。这在我们的安装中提供了大约1750 MB 
 //内存。 
 long realGrowN = 166608000; // 
 for（int i = 0; i< realGrowN; i ++）{
 list.add（23L）; 
} 
 
 System.out.println（内存分配/增长 - 在垃圾收集前休眠）; 
 Thread.sleep（10 * 1000）; 
 
 list = null; 
 System.gc（）; 
 
 System.out.println（垃圾收集 - 永远沉睡）; 
 while（true）{
 Thread.sleep（1 * 1000）; 
} 
} 
} 
  

版本：

 > java -version 
 openjdk版本1.8.0_66-internal
 OpenJDK运行时环境（build 1.8.0_66-internal-b01）
 OpenJDK 64位服务器VM（版本25.66-b01，混合模式）
 
> uname -a 
 Linux londo 3.16.0-4-amd64＃1 SMP Debian 3.16.7-ckt11-1 + deb8u5（2015-10-09）x86_64 GNU / Linux 
 
> lsb_release -a 
没有LSB模块可用。 
经销商ID：Debian 
描述：Debian GNU / Linux 8.2（jessie）
发布：8.2 
代号：jessie 
  

我也尝试过OpenJDK 1.7和Sun Java 1.8。所有行为都相似，并且没有
将内存返回给操作系统。

我认为我需要这个并且交换和分页不会解决这个，因为在寻呼上花费磁盘IO接近2GB垃圾进出只是浪费资源。如果您不同意，请赐教。

我还写了一些带有 malloc（）的memoryUsage.c / free（）， 将内存返回给操作系统。所以C中可能可能。也许不是用Java？

编辑：Augusto指出搜索会导致我 -XX：MaxHeapFreeRatio 和 -XX：MinHeapFreeRatio 仅适用于 -XX：+ UseSerialGC 。我欣喜若狂，尝试了一下，感到困惑的是我自己没有找到这个。是的，它确实适用于我的MemoryUsage.java：

然而，当我尝试 -XX：+ UseSerialGC 时我们真正的应用程序，而不是：

我发现gc（）一段时间后做了帮助，所以我创建了一个线程或多或少：

  while（idle（）&& memoryTooLarge（）&&！tooManyAttemptsYet（））{ 
 Thread.sleep（10 * 1000）; 
 System.gc（）; 
} 
  

这就是诀窍：

I实际上以前曾看到过 -XX：+ UseSerialGC 和多个 System.gc（）的行为。许多实验，但不喜欢GC线程的需要。谁知道，随着我们的app和java的发展，这种情况是否会继续发挥作用。必须有更好的方法。

强制我调用的逻辑是什么 System.gc（）四时间（但不是立即），这些东西记录在哪里？

搜索 -XX的文档：MaxHeapFreeRatio 和 -XX：MinHeapFreeRatio 仅使用 -XX：+ UseSerialGC ，我读了 java工具/可执行文件的文档，并且在 -XX：MaxHeapFreeRatio 和 -XX：MinHeapFreeRatio 仅适用于 -XX：+ UseSerialGC 。事实上，已修复的问题 [JDK-8028391]使Min / MaxHeapFreeRatio标志易于管理说：

为了让应用程序能够控制如何以及何时允许更多或更少的GC，
标志-XX ：MinHeapFreeRatio和-XX：MaxHeapFreeRatio应该是
可管理的。还应在
默认并行收集器中实现对这些标志的支持。

A comment 对于已修复的问题说：

这些标志的支持也被添加到ParallelGC中作为
的一部分适应规模政策。

我已经检查了，补丁在已修复的问题已向后移植到openjdk-8 确实包含在源包tarball 。所以它显然应该在默认的并行收集器中工作，但不像我在这篇文章中所证明的那样。我还没有找到任何文档说它只适用于 -XX：+ UseSerialGC 。正如我在这里记录的那样，即使这是不可靠的/冒险的。

我不能得到 -XX：MaxHeapFreeRatio 和 -XX：MinHeapFreeRatio 做他们承诺的事情而不必经历所有这些箍？

解决方案

G1（-XX：+ UseG1GC），并行清除（-XX：+ UseParallelGC）和
ParallelOld（-XX：+ UseParallelOldGC）返回堆
收缩时的内存。我不太确定Serial和CMS，他们在我的实验中没有收缩
。

两个并行收集器都需要一些GC在将
缩减到可接受的大小之前。这是每个设计。它们是b $ b b故意保留在堆上，假设将来需要
。设置标志-XX：GCTimeRatio = 1会稍微改善
的情况，但仍需要几个GC缩减很多。

G1非常擅长快速缩小堆，所以对于上面描述的用例
我会说它可以通过使用G1并在释放后运行
System.gc（）来解决所有缓存和类加载器等。

http://bugs.java.com/bugdatabase/view_bug.do?bug_id=6498735

We have a long-lived server process, that infrequently needs a lot of RAM for a short while. We see that once the JVM has gotten the memory from the OS, it never returns it back to the OS. How do we ask the JVM to return heap memory back to the OS?

Typically, the accepted answer to such questions is to use -XX:MaxHeapFreeRatio and -XX:MinHeapFreeRatio. (See e.g. 1,2,3,4). But we're running java like this:

java -Xmx4G -XX:MaxHeapFreeRatio=50 -XX:MinHeapFreeRatio=30 MemoryUsage

and still see this in VisualVM:

Clearly, the JVM is not honoring -XX:MaxHeapFreeRatio=50 as the heapFreeRatio is very close to 100% and nowhere near 50%. No amount of clicking on "Perform GC" returns memory to the OS.

MemoryUsage.java:

import java.util.ArrayList;
import java.util.List;

public class MemoryUsage {

    public static void main(String[] args) throws InterruptedException {
        System.out.println("Sleeping before allocating memory");
        Thread.sleep(10*1000);

        System.out.println("Allocating/growing memory");
        List<Long> list = new ArrayList<>();
        // Experimentally determined factor. This gives approximately 1750 MB
        // memory in our installation.
        long realGrowN = 166608000; //
        for (int i = 0 ; i < realGrowN ; i++) {
            list.add(23L);
        }

        System.out.println("Memory allocated/grown - sleeping before Garbage collecting");
        Thread.sleep(10*1000);

        list = null;
        System.gc();

        System.out.println("Garbage collected - sleeping forever");
        while (true) {
            Thread.sleep(1*1000);
        }
    }
}

Versions:

> java -version
openjdk version "1.8.0_66-internal"
OpenJDK Runtime Environment (build 1.8.0_66-internal-b01)
OpenJDK 64-Bit Server VM (build 25.66-b01, mixed mode)

> uname -a
Linux londo 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt11-1+deb8u5 (2015-10-09) x86_64 GNU/Linux

> lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux 8.2 (jessie)
Release:    8.2
Codename:   jessie

I've also tried OpenJDK 1.7 and Sun Java's 1.8. All behave similarly and none give memory back to the OS.

I do think I need this and that swap and paging won't "solve" this, because spending disk IO on paging close to 2GB garbage in and out is just a waste of resources. If you disagree, please enlighten me.

I've also written a little memoryUsage.c with malloc()/free(), and it does return memory to the OS. So it is possible in C. Perhaps not with Java?

Edit: Augusto pointed out that searching would've led me to -XX:MaxHeapFreeRatio and -XX:MinHeapFreeRatio only worked with -XX:+UseSerialGC. I was ecstatic and tried it, puzzled that I hadn't found this myself. Yes, it did work with my MemoryUsage.java:

However, when I tried -XX:+UseSerialGC with our real app, not so much:

I discovered that gc() after a while did help, so I made a thread that did more or less:

while (idle() && memoryTooLarge() && ! tooManyAttemptsYet()) {
    Thread.sleep(10*1000);
    System.gc();
}

and that did the trick:

I had actually previously seen the behavior with -XX:+UseSerialGC and multiple System.gc() calls in some of my many experiments but didn't like the need for a GC thread. And who knows if that'll continue to work as both our app and java evolves. There must be a better way.

What is the logic that forces me to call System.gc() four times (but not immediately), and where is this stuff documented?

In search of documentation for -XX:MaxHeapFreeRatio and -XX:MinHeapFreeRatio only working with -XX:+UseSerialGC, I read the documentation for the java tool/executable and it isn't mentioned anywhere that -XX:MaxHeapFreeRatio and -XX:MinHeapFreeRatio only works with -XX:+UseSerialGC. In fact, the fixed issue [JDK-8028391] Make the Min/MaxHeapFreeRatio flags manageable says:

To enable applications to control how and when to allow for more or less GC, the flags -XX:MinHeapFreeRatio and -XX:MaxHeapFreeRatio should be made manageable. Support for these flags should also be implemented in the default parallel collector.

A comment for the fixed issue says:

Support for these flags have also been added to the ParallelGC as part of the adaptive size policy.

I've checked, and the patch referenced in the fixed issue backported to openjdk-8 is indeed contained in the source package tarball for the openjdk-8 version I'm using. So it should apparently work in "the default parallel collector", but doesn't as I've demonstrated in this post. I haven't yet found any documentation that says it should only work with -XX:+UseSerialGC. And as I've documented here, even this is unreliable/dicey.

Can't I just get -XX:MaxHeapFreeRatio and -XX:MinHeapFreeRatio to do what they promise without having to go through all these hoops?

解决方案

G1 (-XX:+UseG1GC), Parallel scavenge (-XX:+UseParallelGC) and ParallelOld (-XX:+UseParallelOldGC) do return memory when the heap shrinks. I'm not so sure about Serial and CMS, they didn't shrink their heap in my experiments.

Both parallel collectors do require a number of GCs before shrinking the heap down to an "acceptable" size. This is per design. They are deliberately holding on to the heap assuming that it will be needed in the future. Setting the flag -XX:GCTimeRatio=1 will improve the situation somewhat but it will still take several GCs to shrink a lot.

G1 is remarkably good at shrinking the heap fast, so for the usecase described above I would say it's solvable by using G1 and running System.gc() after having released all caches and class loaders etc.

http://bugs.java.com/bugdatabase/view_bug.do?bug_id=6498735

这篇关于OpenJDK JVM是否会将堆内存返回给Linux？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！