Java进程内存使用量不断增加 [英] Java process memory usage keeps increasing infinitely

问题描述

前提条件：

• 装有16 Gb内存的PC


• 安装了JDK 1.8.x Ubuntu 16.10 x64。


• 标准的基于Spring的Web应用程序，部署在Tomcat 8.5.x上。 Tomcat配置了下一个参数： CATALINA_OPTS =$ CATALINA_OPTS -Xms128m -Xmx512m -XX：NewSize = 64m -XX：MaxNewSize = 128m -Xss512k -XX：+ UseParallelGC -XX：+ AggressiveOpts -XX：+ UseFastAccessorMethods -XX：MaxMetaspaceSize = 512m -XX：-TieredCompilation -XX：ReservedCodeCacheSize = 512m


• JMeter 2.13用于运行负载测试


• JProfiler 9.x for java heap memory usage tracking


• top util for java process memory usage tracking

当我按顺序开始加载测试3次时，我观察到（使用 top ）java进程是增加一些已用内存：

• 在Tomcat启动后，在第一次测试后使用~1Gb


• 所有测试完成后运行它使用4.5Gb


• Tomcat使用7Gb RAM

所有这些时间堆大小都是有限的，JProfiler确认 - 堆大小不超过512Mb。

这是一个碎石JProfiler的侄女。底部的红色数字是java进程使用的内存大小（根据 top ）。

问题是：为什么java进程会一直增加内存使用量它有效吗？

谢谢！

UPD＃1：关于可能的重复：它们已确认这只发生在Solaris上。但我使用的是Ubuntu 16.10。同样，尖锐的问题没有解释问题原因的答案。

UPD＃2：我暂停了一段时间后不得不回到这个问题。现在我使用 pmap util来转储 java 进程使用的内存。我有三个转储：在测试运行之前，在第一次测试执行之后和在执行一些N次测试之后。测试它们会为应用程序产生大量流量。所有转储都在这里：

看起来我的问题与此处描述的几乎相同：
然后我做了相同的负载测试，但使用Apache CXF实现，它给出了下一个结果：
所以你可以看到CXF使用更少的内存并且它更稳定（它并没有增长所有内存）时间为ref.impl。）。
最后我在JDK问题跟踪器上发现了一个问题 - https：// bug。 openjdk.java.net/browse/JDK-8074108 - 这又是关于zip库中的内存泄漏问题，而且问题尚未解决。所以看起来我无法真正解决我的应用中的内存泄漏问题，只是可以做一些解决方法。

谢谢大家的帮助！

解决方案

我的假设是你在JProfiler中收集分配信息/调用栈/等，你观察到的RSS增长与JProfiler有关，将数据保存在内存中。

您可以通过收集更少的信息来验证这是否属实（在分析开始时应该有一个屏幕，允许您例如不收集对象分配）并查看是否因此，您会观察到较小的RSS增长。在没有JProfiler的情况下运行负载测试也是一种选择。

我有一个类似案例。

Preconditions:

• PC with 16 Gb of RAM
• JDK 1.8.x installed on Ubuntu 16.10 x64.
• a standard Spring-based web application, that is deployed on Tomcat 8.5.x. Tomcat is configured with next parameters: CATALINA_OPTS="$CATALINA_OPTS -Xms128m -Xmx512m -XX:NewSize=64m -XX:MaxNewSize=128m -Xss512k -XX:+UseParallelGC -XX:+AggressiveOpts -XX:+UseFastAccessorMethods -XX:MaxMetaspaceSize=512m -XX:-TieredCompilation -XX:ReservedCodeCacheSize=512m"
• JMeter 2.13 for load tests running
• JProfiler 9.x for java heap memory usage tracking
• top util for java process memory usage tracking

When I start load tests sequentially 3 times I observe (using top) that java process is increasing a number of used memory:

• after Tomcat start it uses ~1Gb
• after first test running it uses 4.5Gb
• when all tests are finished Tomcat is using 7Gb of RAM

The all this time heap size is limited and JProfiler confirms that - heap size does not exceed 512Mb.

This is a screenshot of JProfiler. Red numbers at the bottom are memory size is used by java process (according to top).

The question is: why does the java process keep increasing memory usage the all time while it's working?

Thanks!

UPD#1: About the possible duplicate: they have confirmed that this only happens on Solaris. but I use Ubuntu 16.10. As well the pointed question does not have an answer that would explain the cause of the problem.

UPD#2: I had to return to this issue after some pause. And now I use pmap util to make a dump of memory used by the java process. I have three dumps: before tests running, after the first tests execution and after some N tests executions. Tests they produce a lot of traffic to the application. All dumps are here: https://gist.github.com/proshin-roman/752cea2dc25cde64b30514ed9ed9bbd0. They are quite huge but the most interesting things are on the 8th line with size of heap: it takes 282.272 Kb before tests and 3.036.400 Kb finally - more than 10x difference! And it's growing each time I run tests. At the same time the heap size is constant (according to JProfiler/VisualVM). What options do I have to find the cause of this problem? Debug JVM? I've tried to find any ways to "look" at this segment of memory but failed. So:

• can I identify somehow content of the [heap] segment of memory?
• does such behavior of java look expected?

I will appreciate any tips about this problem. Thanks all!

UPD #3: using jemalloc (thanks @ivan for the idea) I got next image:

And it looks like I have almost the same problem as described here: http://www.evanjones.ca/java-native-leak-bug.html

UPD #4: for now I found that the issue is related to java.util.zip.Inflater/Deflater and these classes are used in many places in my application. But the largest impact on memory consumption makes interaction with remove SOAP-service. My application uses reference implementation of JAX-WS standard and it gave next memory consumption under load (it has low precision after 10Gb): Then I've made the same load tests but with Apache CXF implementation and it gave next result: So you can see that CXF uses less memory and it's more stable (it's not growing the all time as ref.impl.). Finally I found an issue on JDK issue tracker - https://bugs.openjdk.java.net/browse/JDK-8074108 - it's again about memory leaks in zip library and the issue is not closed yet. So it looks like I can not really fix the problem with memory leaks in my app, just can make some workaround.

Thanks all for your help!

解决方案

My hypothesis is that you collect allocation info / call stacks / etc in JProfiler and the RSS growth you observe is related to JProfiler keeping that data in memory.

You can verify if that's true by collecting less info (there should be a screen at the start of the profiling allowing you to e.g. not collect object allocations) and see if you observe smaller RSS growth as a result. Running your load test without JProfiler is also an option.

I had a similar case in the past.

这篇关于Java进程内存使用量不断增加的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！