调整G1GC参数以进行积极的垃圾收集吗? [英] Tuning G1GC params for aggressive garbage collection?

问题描述

我的JVM基本上是一个火花执行器，它一个接一个地运行任务.一项任务非常耗内存，并且在其生命周期中需要大量内存.

My JVM is basically a spark executor which is running tasks one after another. A task is memory hungry and requires significant memory during its lifecycle.

JConsole和JVisualVM并排报告

以上JVM在带有默认参数的G1GC上运行.正如您在右侧4:25到4:32 PM之间的VisualVM报告中所看到的，峰值是由于执行者运行的每个任务引起的(本质上，每个峰值都是由于执行者在执行任务之后选择了新任务而造成的).前一个已完成).当我在4:35触发手动GC时，我看到堆使用量急剧下降.另外，正如您在JConsole报告的左侧所看到的那样，G1GC从未收集过旧基因世代空间(16:35之前的旧基因世代空间急剧下降是由于手动GC).

因为我的应用程序是一个批处理作业应用程序，所以如果JVM花大量时间进行GC，我可以.但是，我的内存不足.因此，我想知道如何调整JVM G1GC参数，以便有更频繁的GC(也收集了旧的Gen空间)，并且可以用更少的堆空间(XMX)来完成工作.

The above JVM is running on G1GC with default params. As you can see in the VisualVM report on the right hand side between 4:25 to 4:32 PM, the spikes are due to each tasks run by the executor (essentially, each spike is due to the executor picking up new task after the previous one is finished). When I triggered a manual GC at 4:35, I saw a sharp decline the heap usage. Also, as you can see on the left hand side in the JConsole report, the old gen space is never collected by G1GC (the sharp decline in old gen space just before 16:35 is due to the manual GC).

As my application is a spark batch job application, I am ok if the JVMs spent good amount of time doing GC. But, I am running bit short on memory. So, I wanted to know how I can tune my JVM G1GC params so that there are more frequent GC (with the old gen space also getting collected) and I can get the work done with considerably lesses heap space (XMX).

推荐答案

对于G1，OpenJDK 12功能是迅速将未使用的内存返回给操作系统:

For G1, promptly returning unused memory to the operating system is an OpenJDK 12 feature:

• JEP 346:立即从G1返回未使用的承诺内存 如果内存不足，则可能需要控制RSS(从操作系统分配的总内存)，而不仅仅是Java堆中已使用的内存.在当前的OpenJDK中使用G1时，(半)手动触发的完整GC似乎是触发RSS减少的唯一方法. (如果您更改经常发生完整GC的人机工程学，G1的性能将很差.)

  If you a short on memory, it's likely that you need to keep RSS (total memory allocated from the operating system) under control, not just used memory in the Java heap. With G1 in current OpenJDK, (semi-)manually triggered full GCs appear to be the only way to trigger RSS reduction. (G1 will perform poorly if you change the ergonomics that full GCs happen frequently.)

  如果您的OpenJDK版本包含 Shenondoah收集器，则可能是如果您需要节省内存，尤其是进行一些调整(例如使用-XX:ShenandoahGCHeuristics=compact.

  If your OpenJDK build contains the Shenondoah collector, it may be a better choice if you need to conserve memory, particularly with some tuning, such as using -XX:ShenandoahGCHeuristics=compact.

  ZGC 最终也应该支持返回内存，但是它目前没有； 添加-XX:+ZReleaseUnusedHeap选项的补丁尚未被合并.

  ZGC should eventually support returning memory as well, but it currently does not; the patch adding an -XX:+ZReleaseUnusedHeap option has not been merged.

  这篇关于调整G1GC参数以进行积极的垃圾收集吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！