Java速度慢且堆大 [英] Java slower with big heap

问题描述

我有一个在(大)图上运行的Java程序.因此，它使用了大量的堆空间(约50GB，约占主机物理内存的25％).一方面，程序(反复地)从图中选择一个节点并对其进行一些计算.对于某些节点，此计算所花费的时间比预期的要长得多(30-60分钟，而不是预期的几秒钟).为了剖析这些操作以找出耗时的时间，我创建了一个测试程序，该程序仅创建大型图的一小部分，然后在需要花费很长时间进行计算的节点之一上运行相同的操作原始程序.因此，与原始程序相比，测试程序显然只使用了很少的堆空间.

I have a Java program that operates on a (large) graph. Thus, it uses a significant amount of heap space (~50GB, which is about 25% of the physical memory on the host machine). At one point, the program (repeatedly) picks one node from the graph and does some computation with it. For some nodes, this computation takes much longer than anticipated (30-60 minutes, instead of an expected few seconds). In order to profile these opertations to find out what takes so much time, I have created a test program that creates only a very small part of the large graph and then runs the same operation on one of the nodes that took very long to compute in the original program. Thus, the test program obviously only uses very little heap space, compared to the original program.

事实证明，在原始程序中花费48分钟的操作可以在测试程序中在9秒内完成.这真的让我感到困惑.第一个想法可能是较大的程序在垃圾回收上花费了大量时间.因此，我打开了VM垃圾收集器的详细模式.因此，在48分钟内没有执行完整的垃圾收集，而在年轻一代中仅进行了大约20次收集，每个收集所用的时间不到1秒.

It turns out that an operation that took 48 minutes in the original program can be done in 9 seconds in the test program. This really confuses me. The first thought might be that the larger program spends a lot of time on garbage collection. So I turned on the verbose mode of the VM's garbage collector. According to that, no full garbage collections are performed during the 48 minutes, and only about 20 collections in the young generation, which each take less than 1 second.

所以我的问题是，还有什么可以解释如此巨大的时间差异?我不太了解Java在内部如何组织堆.对于具有大量活动对象的大型堆，是否需要花费更长的时间?在这样的设置下，对象分配是否可能会花费更长的时间，因为在堆中找到适当的位置会花费更长的时间?还是VM会对堆进行内部重组，而这可能会花费很多时间(显然，除了垃圾回收之外).

So my questions is what else could there be that explains such a huge difference in timing? I don't know much about how Java internally organizes the heap. Is there something that takes significantly longer for a large heap with a large number of live objects? Could it be that object allocation takes much longer in such a setting, because it takes longer to find an adequate place in the heap? Or does the VM do any internal reorganization of the heap that might take a lot of time (besides garbage collection, obviously).

如果有任何重要意义，我正在使用Oracle JDK 1.7.

I am using Oracle JDK 1.7, if that's of any importance.

推荐答案

虽然更大的内存可能意味着更大的问题，但我会说(除了您已排除的GC外)没有什么可以将9秒延长到48分钟( 320).

While bigger memory might mean bigger problems, I'd say there's nothing (except the GC which you've excluded) what could extend 9 seconds to 48 minutes (factor 320).

大堆似乎使空间局部性变差，但我认为这并不重要.我不同意蒂姆的回答. "必须保留所有内容的缓存".

A big heap makes seemingly worse spatial locality possible, but I don't think it matters. I disagree with Tim's answer w.r.t. "having to leave the cache for everything".

还有 TLB ，该缓存用于虚拟地址转换，这可能会引起一些问题具有非常大的内存.但同样，不是320因子.

There's also the TLB which a cache for the virtual address translation, which could cause some problems with very large memory. But again, not factor 320.

我认为JVM中没有任何东西可能导致此类问题.

I don't think there's anything in the JVM which could cause such problems.

我能想象的唯一原因是，尽管您有足够的物理内存，但仍有一些交换空间被使用.甚至很小的交换也可能是造成严重减速的原因.确保它已关闭(并可能检查 Swappiness ).

The only reason I can imagine is that you have some swap space which gets used - despite the fact that you have enough physical memory. Even slight swapping can be the cause for a huge slowdown. Make sure it's off (and possibly check swappiness).

这篇关于Java速度慢且堆大的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！