调整巨额阵列 [英] Resizing huge arrays

问题描述

我在我的应用程序处理庞大的阵列，需要调整它们的大小。

I'm dealing with huge arrays in my application and need to resize them.

假设你拥有2GB的数组，你想它调整为3Gb。有没有一种方法来调整它，而不需要暂时5Gb的？

Let's say you have an array of 2Gb and you want to resize it to 3Gb. Is there a way to resize it without needing temporarily 5Gb?

例如，考虑使用 -Xmx1G 标志1Gb的堆：

For instance, given a 1Gb heap using the -Xmx1G flag:

public class Biggy {
    public static void main(String[] args) {
        int[] array;

        array = new int[100 * 1000 * 1000]; // needs 400Mb, works
        array = null; // needed for GC
        array = new int[150 * 1000 * 1000]; // needs 600Mb, works
        array = null; // needed for GC
        array = new int[100 * 1000 * 1000]; // needs 400Mb, works
        array = Arrays.copyOf(array, 150 * 1000 * 1000); // needs 1000Mb, throws out of memory
    }
}

那么，有没有一种方法来调整阵列，而不需要额外的临时内存？

So, is there a way to resize arrays without requiring that extra temporary memory?

推荐答案

我会用一个List＆LT; INT []>中的每个int []大小是固定的。例如1.28亿。为了发展整个收藏只涉及添加另一个数组。我用IntBuffer在其避免了需要使用堆直接存储器。 （或使用内存映射文件，这意味着它不使用堆或直接存储器;）这是丑，我使用包装类隐藏的丑陋。它很好地执行pretty。对于内存映射文件，我可以用一个阵列，这是比物理内存更大。

I would use a List<int[]> where each int[] is a fixed size. e.g. 128 million. To grow the whole "collection" only involves added another array. I use IntBuffer in direct memory which avoids the need to use heap. (or use memory mapped files which means it doesn't use heap or direct memory ;) This is ugly, and I use a wrapper class to hide the ugliness. It does perform pretty nicely. With memory mapped files I can use an "array" which is larger than the physical memory.

private final List<IntBuffer> array = new ArrayList<IntBuffer>();

public int get(long n) {
    return array.get((int)(n >> 27)).get(n & ((1 << 27) -1));
}

public void put(long n, int v) {
    return array.get((int)(n >> 27)).put(n & ((1 << 27) -1), v);
}

这篇关于调整巨额阵列的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！