Java：内存高效的ByteArrayOutputStream [英] Java: Memory efficient ByteArrayOutputStream

问题描述

我在磁盘中有一个40MB的文件，我需要使用字节数组将其映射到内存中。

I've got a 40MB file in the disk and I need to "map" it into memory using a byte array.

起初，我想写的是文件到ByteArrayOutputStream是最好的方法，但我发现在复制操作期间的某个时刻需要大约160MB的堆空间。

At first, I thought writing the file to a ByteArrayOutputStream would be the best way, but I find it takes about 160MB of heap space at some moment during the copy operation.

有人知道更好的方法吗不使用RAM文件大小的三倍这样做吗？

Does somebody know a better way to do this without using three times the file size of RAM?

更新：感谢您的回答。我注意到我可以减少内存消耗，稍微告诉ByteArrayOutputStream的初始大小比原始文件大小稍大一些（使用我的代码强制重新分配的确切大小，检查原因）。

Update: Thanks for your answers. I noticed I could reduce memory consumption a little telling ByteArrayOutputStream initial size to be a bit greater than the original file size (using the exact size with my code forces reallocation, got to check why).

还有另一个高内存点：当我用ByteArrayOutputStream.toByteArray返回byte []时。看看它的源代码，我可以看到它正在克隆数组：

There's another high memory spot: when I get byte[] back with ByteArrayOutputStream.toByteArray. Taking a look to its source code, I can see it is cloning the array:

public synchronized byte toByteArray()[] {
    return Arrays.copyOf(buf, count);
}

我想我可以扩展ByteArrayOutputStream并重写此方法，所以要直接返回原始数组。这里有没有潜在的危险，因为流和字节数组不会被多次使用？

I'm thinking I could just extend ByteArrayOutputStream and rewrite this method, so to return the original array directly. Is there any potential danger here, given the stream and the byte array won't be used more than once?

推荐答案

MappedByteBuffer 可能就是你要找的东西。

MappedByteBuffer might be what you're looking for.

我很惊讶它需要这么多RAM来读取内存中的文件。您是否构建了具有适当容量的 ByteArrayOutputStream ？如果还没有，则流可以在接近40 MB的末尾时分配一个新的字节数组，这意味着您将拥有一个39 MB的完整缓冲区和两倍大小的新缓冲区。然而，如果流具有适当的容量，则不会有任何重新分配（更快），也不会浪费内存。

I'm surprised it takes so much RAM to read a file in memory, though. Have you constructed the ByteArrayOutputStream with an appropriate capacity? If you haven't, the stream could allocate a new byte array when it's near the end of the 40 MB, meaning that you would, for example, have a full buffer of 39MB, and a new buffer of twice the size. Whereas if the stream has the appropriate capacity, there won't be any reallocation (faster), and no wasted memory.

这篇关于Java：内存高效的ByteArrayOutputStream的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！