有没有办法强制JVM使用swap，无论内存需求有多大？ [英] Is there any way to force a JVM to use swap no matter how big the memory requirement is?

问题描述

这是我的情况：我手头有一项需要大量记忆的任务。我没有足够的ram，无论我尝试了什么（Jrockit和/ 3gb交换机等），我都不能给JVM足够的ram并且操作以异常终止，告诉我需要更多的堆空间。

Here is my situation: I have a task at hand that requires lots of memory. I do not have enough ram, and no matter what I tried (Jrockit with /3gb switch etc), I can't give JVM enough ram and the operation is terminated with an exception, telling me I need more heap space.

有什么办法可以强迫JVM使用操作系统的交换机制，这样它就不会耗尽内存？这是Windows xp 32位

Is there any way I can force the JVM to use the OS's swapping mechanism so that it won't run out of memory? This is Windows xp 32 bit

这需要很长时间，但我不在乎，我只需要完成此操作。

It would take ages, but I would not care, I just need this operation to be completed.

我的选项用完了，而且我无法控制任何变量..

I've run out of options, and I have no control over any of the variables here..

这是一个必需的编辑，因为我几乎每个人都有相同的响应:)
这不是我的代码。有人编写了一个将xml文件读入存储库的工具。该工具使用EMF，并立即加载整个模型。我所能做的就是为它提供XML文件。
如果本机代码在Windows或Linux等下运行，操作系统使用虚拟内存/交换空间为其提供内存，应用程序不知道它。
我想知道是否可以对JVM做同样的事情。在Windows 32位下，-Xmx可以达到一定数量，但这还不够。
目前，出门购买新硬件不是我的选择。所以我想知道是否有可能使JVM像本机进程一样工作。慢，但仍在工作。
显然这是不可能的，我运气不好。我只需要知道我是否真的没有选择。

This is a required edit, since I am having the same response from pretty much everyone :) This is not my code. Someone has written a tool that reads an xml file into a repository. The tool uses EMF, and loads the whole model at once. All I can do is to feed it the XML file. In case of native code running under Windows or Linux etc, the OS provides memory to it, using virtual memory/swap space, and the app does not know about it. I was wondering if it is possible to do the same with the JVM. Under Windows 32 bit, -Xmx can go up to a certain amount, but that is not enough. Going out and buying new hardware is not an option for me for the moment. So I was wondering if it is possible to make the JVM work like native processes. Slow, but still working. Apparently that is not possible, and I am out of luck. I just need to know if I'm really out of options.

推荐答案

显然有一种方法可以绕过Java堆的极限。它甚至被用于名为 BigMemory 的商业产品中，它基本上允许您通过透明地交换来获得几乎无限的内存操作系统交换和/或磁盘（如果需要）。

Apparently there is one way around the limits of Java heap. It is even used in a commercial product called BigMemory which basically allows you to have almost unlimited memory by transparently swapping out to OS swap and/or to disk if needed.

想法是使用直接 ByteBuffer 来存储你的对象数据。因为直接字节缓冲区的内容存储在本机进程内存中（而不是堆），所以可以依靠操作系统交换机制为您交换内存。我在这个网站上找到了这一点（在网站上搜索直接字节缓冲区）页面）。

The idea is to use direct ByteBuffers to store your objects data. Because direct byte buffers' contents are stored in native process memory (as opposed to heap) you can rely on OS swap mechanism to swap memory out for you. I found this on this website (search for 'direct byte buffer' on the page).

以下是如何实现它（java-pseudo-code'ish）：

Here is how you can implement it (java-pseudo-code'ish):

class NativeMemoryCache{
  private Map<Object, ByteBuffer> data = new HashMap<...>();

  public void put(Object key, Serializable object){
    byte[] bytes = serialize(object);
    //allocate native memory to store our object
    ByteBuffer buf = ByteBuffer.allocateDirect(bytes.length);
    buf.put(bytes);
    buf.flip();
    data.put(key, buf);
  }

  public Object get(Object key){
    ByteBuffer buf = data.get(key).duplicate();
    byte[] bytes = new byte[buf.remaining()];
    buf.get(bytes);
    return deserialize(bytes);
  }

  private byte[] serialize(Object obj){ ... }
  private Object deserialize(byte[] bytes){ ... }
}

希望你能理解。你只需要实现序列化（你也可以使用zip来压缩你的对象。如果你有很少的大对象，特别是包含像字符串这样的可拉伸数据的对象，这将是有效的。）

Hope you get the idea. You just need to implement the serialization (you can also compress your objects using zip. This will be effective if you have few big objects especially ones containing zippable data like strings).

当然 NativeMemoryCache 对象，数据哈希映射和密钥 s将在堆中，但这不应该占用太多内存。

Of course NativeMemoryCache object, data hash map and keys will be in heap, but that should not take much memory.

这篇关于有没有办法强制JVM使用swap，无论内存需求有多大？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！