Java程序运行一段时间后会变得越来越慢 [英] Java program is getting slower after running for a while

问题描述

我有一个java程序，它是一个典型的机器学习算法，通过一些等式更新一些参数的值：

  for（int iter = 0; iter< 1000; iter ++）{
 // 1.创建许多临时变量并执行一些计算
 // 2.更新参数
}的值
  

更新参数的计算相当复杂，我必须创建许多临时对象，但它们不是在循环中引用。循环中的代码是CPU密集型的，并且不访问磁盘。这个程序加载了一个相对较大的训练数据集，因此，我向JVM授予了10G内存（-Xmx10G），这比它需要的大得多（通过top命令或窗口的任务管理器在〜6G的峰值）。

我在几台安装了SUN热点JDK / JRE 1.8的Linux机器（centos 6,24G内存）和一台窗口机器（win7,12G）上进行了测试。我没有指定除-Xmx之外的其他JVM参数。这两款机器都是专门用于我的程序的。

在windows上，我的程序运行良好：每次迭代使用非常相似的运行时间。但是，所有centos机器的运行时间都很奇怪。
它最初运行正常，但在第7次/第8次迭代时显着减速（减慢约10倍），然后在每次迭代中保持减速〜10％。

我怀疑它可能是由Java的垃圾收集器引起的。因此，我使用jconsole来监视我的程序。次要GC在两台机器上都经常发生，这是因为程序在循环中创建了许多临时变量。此外，我使用jstat -gcutil $ pid $ 1s命令并获取了统计信息：

Centos： https://www.dropbox.com/s/ioz7ai6i1h57eoo/jstat.png?dl=0

窗口： https：// www.dropbox.com/s/3uxb7ltbx9kpm9l/jstat-winpng.png?dl=0

但是，两种机器的统计数字不同的是：

1. 窗口上的S1在0到50之间快速跳转，而在centos上保持0.00。


2. 窗口上的E在0到100之间变化非常迅速。当我每秒打印一次统计数据时，屏幕截图不会将其增量显示为100.但在centos中，E会增加缓慢地向100，然后减少到0，并再次增加。
   

我的程序的怪异行为似乎是由于到Java GC？我是Java性能监视器的新手，对于优化GC参数设置没有好处。你有什么建议吗？非常感谢！

解决方案

给Java（或任何垃圾收集语言）过多的内存会对性能产生不利影响。实时（被引用的）对象在内存中变得越来越稀疏，导致从主内存中更频繁地获取内存。请注意，在您向我们展示的示例中，更快的窗口比GC更快捷地完成了GC和GC的全局GC操作 - 但GC周期（特别是全部GC）通常对性能不利。

如果运行训练集并不需要很长时间，那么可以尝试使用不同的内存分配基准。

更激进的解决方案，但一个应该有很大影响的是通过回收池中的对象来消除（或尽可能地减少）循环内的对象创建。

I have a java program that is a typical machine learning algorithm, updating the values for some parameters by some equations:

for (int iter=0; iter<1000; iter++) {
    // 1. Create many temporary variables and do some computations                         
    // 2. Update the value for the parameters                    
}

The computations of updating parameters are rather complex, and I have to create many temporary objects, but they are not referenced out of the loop. The code in the loop is CPU-intensive, and does not access disk. This program loads a relatively large training dataset, therefore, I granted 10G memory (-Xmx10G) to JVM, which is much larger than it requires (peak at ~6G by "top" command or window's task manager).

I tested it on several linux machines (centos 6, 24G memory) and a window machine (win7, 12G), both with SUN Hotspot JDK/JRE 1.8 installed. I did not specify other JVM parameters except -Xmx. Both machines are dedicated to my program.

On windows, my program runs well: each iteration uses very similar running time. However, the running time on all of the centos machines is weird. It initially runs properly, but slows down dramatically (~10 times slower) at 7th/8th iteration, and then keeps slow down ~10% in each iteration ever after.

I suspect it might be caused by Java's garbage collector. Therefore, I use jconsole to monitor my program. Minor GC happens very frequently on both machines , that is because the program creates many temporary variable in the loop. Furthermore, I used "jstat -gcutil $pid$ 1s" command and captured the statistics:

Centos: https://www.dropbox.com/s/ioz7ai6i1h57eoo/jstat.png?dl=0

Window: https://www.dropbox.com/s/3uxb7ltbx9kpm9l/jstat-winpng.png?dl=0

[Edited] However, the statistics on two kinds of machines differ a lot:

1. "S1" on windows jumps fast between 0 to 50, while stays at "0.00" on centos.
2. "E" on windows changes very rapidly from 0 to 100. As I print the stat for every second, the screenshot does not show its increment to 100. On centos, however, "E" increases rather slowly towards 100, and then reduces to 0, and increases again.

It seems the weird behaviour of my program is due to Java GC? I am new to Java performance monitor and do not have a good idea to optimize GC parameter setting. Do you have any suggestions? Thank you very much!

解决方案

Giving Java (or any garbage collecting language) too much memory has an adverse effect on performance. The live (referenced) objects become increasing sparse in memory resulting in more frequent fetches from main memory. Note that in the examples you've shown us the faster windows is doing more quick and full GC than Linux - but GC cycles (especially full gcs) are usually bad for performance.

If running the training set does not take an exceptionally long time, then try benchmarking at different memory allocations.

A more radical solution, but one which should have a big impact is to eliminate (or reduce as much as possible) object creation within the loop by recycling objects in pools.

这篇关于Java程序运行一段时间后会变得越来越慢的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！