Java非常大的堆大小 [英] Java very large heap sizes

问题描述

• GC是否使程序无法使用？


• 您使用的是什么GC参数？


• 哪个JVM，Sun或BEA更适合这个？


• 在这种情况下，哪个平台（Linux或Windows）表现更好？


• 在Windows的情况下，64位Vista和XP在这种高内存如果您的应用程序不是交互式的，并且GC暂停不会成为问题对于64位Java来说，处理非常大的堆（即使是数百GB）应该没有任何问题。我们也没有注意到Windows或Linux上的任何稳定性问题。
  
  
  

  然而，当您需要保持GC暂停低时，事情变得非常讨厌：

  
  

  1. 忘记默认吞吐量，停止全球GC。它会暂停您申请中等堆（<〜30 GB）几十秒，大型（>〜30 GB）需要几分钟。购买更快的DIMM不会有帮助。

  最好的选择可能是CMS收集器，由-XX：+ UseConcMarkSweepGC启用。 CMS垃圾收集器仅在初始标记阶段和重新标记阶段停止应用程序。对于非常小的堆如< 4 GB这通常不是问题，但是对于创建大量垃圾和大堆的应用程序来说，重新评估阶段可能需要相当长的时间 - 通常远远少于完全停止世界，但仍然可以是如果CMS垃圾收集器速度不够快，无法在终身代填满之前完成操作，则会回退到标准停止状态，世界的GC。预计大小为16 GB的堆大约需要30秒或更长时间的暂停。您可以尝试避免这种情况，尽可能降低应用程序的长期垃圾生产率。请注意，运行应用程序的内核数量越多，遇到此问题就越大，因为CMS只使用一个内核。显然，要注意不要保证CMS不会退回到STW收集器。而当它发生时，它通常发生在高峰负载，并且你的应用程序已经死了几秒钟。你可能不想为这样的配置签署一个SLA。 那么，这就是新的G1。它的理论设计是为了避免CMS的问题，但我们已经尝试过并观察到：
  
  
  
  
  • 它的吞吐量比CMS差。理论上讲，它应该避免首先收集流行的内存块，然而它很快就会达到几乎所有块都流行的状态，并且它基于的假设只是停止工作。
  
  
  • 最后，G1的停止回退仍然存在;问甲骨文，当代码应该运行。如果他们说从不，问他们为什么代码在那里。所以恕我直言G1真的不会让Java的巨大堆问题消失，它只会使它（可以说）更小一些。
  
  
  
  
  3. 编写应用程序中需要大量C ++内存的该死部分，就像LinkedIn用社交图处理一样。您仍然不会通过这样做避免所有问题（例如堆碎片），但保持较低的停顿肯定更容易。
  
  

  
  

  Does anyone have experience with using very large heaps, 12 GB or higher in Java?

  • Does the GC make the program unusable?
  • What GC params do you use?
  • Which JVM, Sun or BEA would be better suited for this?
  • Which platform, Linux or Windows, performs better under such conditions?
  • In the case of Windows is there any performance difference to be had between 64 bit Vista and XP under such high memory loads?

  解决方案

  If your application is not interactive, and GC pauses are not an issue for you, there shouldn't be any problem for 64-bit Java to handle very large heaps, even in hundreds of GBs. We also haven't noticed any stability issues on either Windows or Linux.

  However, when you need to keep GC pauses low, things get really nasty:

  1. Forget the default throughput, stop-the-world GC. It will pause you application for several tens of seconds for moderate heaps (< ~30 GB) and several minutes for large ones (> ~30 GB). And buying faster DIMMs won't help.

  2. The best bet is probably the CMS collector, enabled by -XX:+UseConcMarkSweepGC. The CMS garbage collector stops the application only for the initial marking phase and remarking phases. For very small heaps like < 4 GB this is usually not a problem, but for an application that creates a lot of garbage and a large heap, the remarking phase can take quite a long time - usually much less then full stop-the-world, but still can be a problem for very large heaps.

  3. When the CMS garbage collector is not fast enough to finish operation before the tenured generation fills up, it falls back to standard stop-the-world GC. Expect ~30 or more second long pauses for heaps of size 16 GB. You can try to avoid this keeping the long-lived garbage production rate of you application as low as possible. Note that the higher the number of the cores running your application is, the bigger is getting this problem, because the CMS utilizes only one core. Obviously, beware there is no guarantee the CMS does not fall back to the STW collector. And when it does, it usually happens at the peak loads, and your application is dead for several seconds. You would probably not want to sign an SLA for such a configuration.

  4. Well, there is that new G1 thing. It is theoretically designed to avoid the problems with CMS, but we have tried it and observed that:

     • Its throughput is worse than that of CMS.
     • It theoretically should avoid collecting the popular blocks of memory first, however it soon reaches a state where almost all blocks are "popular", and the assumptions it is based on simply stop working.
     • Finally, the stop-the-world fallback still exists for G1; ask Oracle, when that code is supposed to be run. If they say "never", ask them, why the code is there. So IMHO G1 really doesn't make the huge heap problem of Java go away, it only makes it (arguably) a little smaller.

  5. If you have bucks for a big server with big memory, you have probably also bucks for a good, commercial hardware accelerated, pauseless GC technology, like the one offered by Azul. We have one of their servers with 384 GB RAM and it really works fine - no pauses, 0-lines of stop-the-world code in the GC.

  6. Write the damn part of your application that requires lots of memory in C++, like LinkedIn did with social graph processing. You still won't avoid all the problems by doing this (e.g. heap fragmentation), but it would be definitely easier to keep the pauses low.

  这篇关于Java非常大的堆大小的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！