如何使JVM使用服务器的最大（所有剩余）内存 [英] How to make JVM use the max (all remain) memory of a server

问题描述

我有一个DFS算法java控制台应用程序，当提供更多内存时运行速度更快。只是一个DFS算法应用程序，既没有I / O也没有其他外部JVM资源使用。它只消耗CPU和内存。该应用程序可以使用1GB内存运行，但使用2 GB内存运行速度要快得多。提供更多内存，应用程序可以运行得更快。我没有触及速度限制为12GB的内存提供。所以我必须使用服务器的所有剩余内存来加速它。并且应用程序不需要并行，一次只能请求一次。

I have an DFS algorithm java console application, that runs faster when more memory is provided. Just a DFS algorithm application, with neither I/O nor other outer-JVM resource usage. It consumes only CPU and memory. The application can run with an 1GB memory, but run much more faster with 2 GB memory. More memory provided, faster the application can run. I haven't touch the speed limit as 12GB of memory provided. So I must use all remain memory of a server to speed it up. And the application need not parallel, one request only at one time.

我需要在不同内存大小的服务器上安装应用程序。

And I need to install the application on different server with different memory size.

有没有办法让JVM使用服务器的所有剩余内存？

Is there a way to let JVM use the all remain memory of the server?

-XX:MaxRAMFraction=1

MaxRAMFraction不是每个服务器都好，有些服务器会导致启动JVM失败位置内存故障，一些工作正常。

MaxRAMFraction is not EVERY server good, some server will result in start JVM failure as location memory failure, some works good.

使用包装器应用程序获取系统保留内存，减去Xmx以外的一些内存使用情况，然后使用相同的Xms启动实际应用程序XMS。该方法还将导致JVM内存分配错误。因为下面的代码返回的内容远远超过我们可以使用的内存，而不仅仅是Xss256m或更多非堆内JVM内存的减号。

Use an wrapper application get system remain memory, and minus some memory usage other than Xmx, then start the real application with same Xms and Xms. The method will also result in JVM memory allocation error. Because the code below returns much more than memory we can use, not just a minus of Xss256m or some more non-heap JVM memory.

com.sun.management.OperatingSystemMXBean mbean = (com.sun.management.OperatingSystemMXBean)
    ManagementFactory.getOperatingSystemMXBean();
long size = mbean.getFreePhysicalMemorySize();

那么让JVM使用服务器的所有剩余内存是否有好方法？

So is there a good way to let JVM use all remain memory of a server?

推荐答案

对于大型内存区域，我使用了堆，这减少了GC的开销，其中一个好处是可以是任何大小的运行时，如果仔细检查，甚至比主内存大。您可以使用直接 ByteBuffer ，但我使用的是我编写的库，它扩展了ByteBuffer功能（>> 2 GB和线程安全） Chronicle Bytes 最大的任何一个使用它是~100 TB的虚拟内存映射到磁盘。

For large regions of memory I use off heap and this reduces overhead on the GC, one of the benefitis is that is can be any size at runtime and even larger than main memory if you do it carefully. You can use direct ByteBuffers but I use a library I wrote which extends the ByteBuffer functionality (>> 2 GB and thread safe) Chronicle Bytes The largest any one uses this is ~100 TB of virtual memory mapped to disk.

我们在Chronicle Bytes之上有两个数据结构，一个键值存储 Chronicle Map 和队列/期刊 Chronicle Queue 。这可以通过更高级别的接口更容易地将数据存储在堆中。

We have two data structures on top of Chronicle Bytes, a key-value store Chronicle Map and a queue/journal Chronicle Queue. This can make storing data off heap easier with a higher level interface.

堆的工作方式，它必须在启动时保留最大堆大小作为单个连续虚拟内存块。特别是，GC假定在清理时随机访问此内存，这意味着如果您的内存略微过度使用，可能是因为在您的内存之后启动了一个进程并且某些堆被换出，您将看到性能急剧下降为你的整个机器。 Windows倾向于开始交换您的GUI，这意味着您无法在没有电源循环的情况下恢复控制。 Linux并没有那么糟糕，但是你现在想要杀死你的进程。这样可以调整它的大小，以便在机器使用情况发生变化时非常难以使用所有内存。

The way the heap works, it has to reserve the maximum heap size on start up as a single continuous block of virtual memory. In particular, the GC assumes random access to this memory on a clean up which means if you are have slightly over utilised your memory, possibly because a process started after yours and some of the heap is swapped out you will see a dramatic fall in performance for your whole machine. Windows tends to start swapping your GUI meaning you can't get back control without a power cycle. Linux isn't as bad, but you will want to kill your process at this point. This makes tuning it size to use all memory very hard if the usage of your machine changes.

通过比较使用虚拟内存，GC不会触及它部分影响不大。您可以将虚拟内存区域多次作为主内存，但只有当前工作集很重要，这在运行时完全由您控制。注意：在Linux上，你的虚拟内存大小可以是你的可用磁盘空间的1000倍，但要小心使用，如果通过触摸太多页面而耗尽，程序将崩溃。

By using virtual memory by comparison, the GC doesn't touch it so unused portions have little impact. You can have areas of virtual memory many times main memory but only your current working set matters, and this is a size entirely in your control at runtime. Note: on Linux you can have virtual memory sizes 1000x your free disk space, but use with care, if you run out by touching too many pages your program will crash.

这篇关于如何使JVM使用服务器的最大（所有剩余）内存的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！