为什么 byte[] 的最大大小是 2 GB - 57 B? [英] Why is the max size of byte[] 2 GB - 57 B?
问题描述
在我的 64 位机器上，这个 C# 代码有效:
On my 64-bit machine, this C# code works:
new byte[2L * 1024 * 1024 * 1024 - 57]
但是这个会抛出一个OutOfMemoryException:
new byte[2L * 1024 * 1024 * 1024 - 56]
为什么?
我知道托管对象的最大大小为 2 GB，并且我创建的数组对象包含的字节数超过了我想要的字节数.即，同步块编号有 4 个字节(或 8 个?)，MethodTable 引用有 8 个字节，数组大小有 4 个字节.那是 24 个字节，包括填充，那么为什么我不能分配一个 2G - 24 个字节的数组呢?最大大小真的恰好 2 GB吗?如果是这样，剩下的 2 GB 用于做什么?
I understand that the maximum size of a managed object is 2 GB and that the array object I'm creating contains more than the bytes I want. Namely, there is 4 bytes (or 8?) for the syncblock number, 8 bytes for MethodTable reference and 4 bytes for the size of the array. That's 24 bytes including padding, so why can't I allocate an array with 2G - 24 bytes? Is the maximum size really exactly 2 GB? If that's the case, what is the rest of 2 GB used for?
注意:我实际上不需要分配一个有 200 万字节的数组.即使我这样做了，56 字节的开销也可以忽略不计.而且我可以使用自定义 BigArray 轻松绕过限制.
Note: I don't actually need to allocate an array with 2 million of bytes. And even if I did, 56 bytes is negligible overhead. And I could easily work around the limit using custom BigArray<T>.
推荐答案
您需要 56 字节的开销.最大尺寸实际上是 2,147,483,649-1 减 56.这就是为什么您的 minus 57 有效而 minus 56 无效.
You need 56 bytes of overhead. It is actually 2,147,483,649-1 minus 56 for the maximum size. That's why your minus 57 works and minus 56 does not.
As 乔恩·斯基特 在这里说:
然而，实际上，我不相信任何实现都支持如此庞大的阵列.CLR 有一个每个对象的限制略低于 2GB，所以即使是字节数组实际上也不能有 2147483648 个元素.一点实验表明，在我的盒子上，您可以创建的最大数组是新字节[2147483591].(那是在64 位 .NET CLR；Mono的版本我已经安装了扼流圈.)
However, in practical terms, I don't believe any implementation supports such huge arrays. The CLR has a per-object limit a bit short of 2GB, so even a byte array can't actually have 2147483648 elements. A bit of experimentation shows that on my box, the largest array you can create is new byte[2147483591]. (That's on the 64 bit .NET CLR; the version of Mono I've got installed chokes on that.)
另见这篇 InformIT 文章相同的主题.它提供了以下代码来演示最大大小和开销:
See also this InformIT article on the same subject. It provides the following code to demonstrate the maximum sizes and overhead:
class Program
{
static void Main(string[] args)
{
AllocateMaxSize<byte>();
AllocateMaxSize<short>();
AllocateMaxSize<int>();
AllocateMaxSize<long>();
AllocateMaxSize<object>();
}
const long twogigLimit = ((long)2 * 1024 * 1024 * 1024) - 1;
static void AllocateMaxSize<T>()
{
int twogig = (int)twogigLimit;
int num;
Type tt = typeof(T);
if (tt.IsValueType)
{
num = twogig / Marshal.SizeOf(typeof(T));
}
else
{
num = twogig / IntPtr.Size;
}
T[] buff;
bool success = false;
do
{
try
{
buff = new T[num];
success = true;
}
catch (OutOfMemoryException)
{
--num;
}
} while (!success);
Console.WriteLine("Maximum size of {0}[] is {1:N0} items.", typeof(T).ToString(), num);
}
}
最后，文章是这样说的:
Finally, the article has this to say:
如果你计算一下，你会看到分配数组的开销是 56 个字节.还剩下一些字节由于对象大小，最后结束.例如 268,435,448 64 位数字占用 2,147,483,584 个字节.添加 56 字节数组开销给你 2,147,483,640，剩下 7字节少 2 GB.
If you do the math, you’ll see that the overhead for allocating an array is 56 bytes. There are some bytes left over at the end due to object sizes. For example, 268,435,448 64-bit numbers occupy 2,147,483,584 bytes. Adding the 56 byte array overhead gives you 2,147,483,640, leaving you 7 bytes short of 2 gigabytes.
等等，还有更多！
环顾四周并与 Jon Skeet 交谈，他向我指出了他在 内存和字符串.在那篇文章中，他提供了一个尺寸表:
Looking around and talking with Jon Skeet, he pointed me to an article he wrote on Of memory and strings. In that article he provides a table of sizes:
Type x86 size x64 size
object 12 24
object[] 16 + length * 4 32 + length * 8
int[] 12 + length * 4 28 + length * 4
byte[] 12 + length 24 + length
string 14 + length * 2 26 + length * 2
先生飞碟接着说:
你可能会因为看上面的数字并认为一个对象的开销"是 12x86 中的字节和 x64 中的 24 字节……但是这不太对.
You might be forgiven for looking at the numbers above and thinking that the "overhead" of an object is 12 bytes in x86 and 24 in x64... but that's not quite right.
还有这个:
在 x86 中每个对象有 8 个字节的基本"开销，每个对象有 16 个字节x64 中的对象...鉴于我们可以在 x86 中存储真实"数据的 Int32并且仍然有一个 12 的对象大小，同样我们可以存储两个 Int32sx64 中的真实数据，并且仍然有一个x64 的对象.
There's a "base" overhead of 8 bytes per object in x86 and 16 per object in x64... given that we can store an Int32 of "real" data in x86 and still have an object size of 12, and likewise we can store two Int32s of real data in x64 and still have an object of x64.
最小"大小分别为 12 字节和 24 字节.在换句话说，你不能有一个类型这只是开销.注意如何空"类占用相同大小作为创建实例对象...实际上有一些备用空间，因为 CLR 没有就像在没有对象的情况下操作数据.(请注意，没有的结构字段也占用空间，即使对于局部变量.)
There's a "minimum" size of 12 bytes and 24 bytes respectively. In other words, you can't have a type which is just the overhead. Note how the "Empty" class takes up the same size as creating instances of Object... there's effectively some spare room, because the CLR doesn't like operating on an object with no data. (Note that a struct with no fields takes up space too, even for local variables.)
x86 对象被填充到 4 字节边界；在 x64 上它是 8 个字节(和以前一样)
The x86 objects are padded to 4 byte boundaries; on x64 it's 8 bytes (just as before)
最后 Jon Skeet 回应了一个 我问他的问题 在他陈述的另一个问题中(回应 InformIT 文章 我给他看了):
and finally Jon Skeet responded to a question I asked of him in another question where he states (in response to the InformIT article I showed him):
看起来像你的文章指的是推断开销只是从极限，这是愚蠢 IMO.
It looks like the article you're referring to is inferring the overhead just from the limit, which is silly IMO.
因此，为了回答您的问题，根据我收集的数据，实际开销是 24 字节，还有 32 字节 的空闲空间.
So to answer your question, actual overhead is 24 bytes with 32 bytes of spare room, from what I gather.
这篇关于为什么 byte[] 的最大大小是 2 GB - 57 B?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
