.Net 中非常大的集合导致内存不足异常 [英] Very large collection in .Net causes out-of-memory exception

问题描述

我正在测试 .Net 中的集合有多大.从技术上讲，任何集合对象都可以增长到物理内存的大小.

I am testing how big a collection could be in .Net. Technically, any collection object could grows to the size of the physical memory.

然后我在一台有 16GB 内存、运行 Windows 2003 服务器和 Visual Studio 2008 的服务器中测试了以下代码.我测试了 F# 和 C# 代码，并在运行时查看了任务管理器.我可以看到，在大约增加 2GB 内存后，程序因内存不足异常而崩溃.我确实在属性页中将目标平台设置为 x64.

Then I tested the following code in a sever, which has 16GB memory, running Windows 2003 server and Visual Studio 2008. I tested both F# and C# code, and looked at the Task Manager while running. I can see that after about growing 2GB memory, the program crashed with out-of-memory exception. I did set the target platform to x64 in the property page.

open System.Collections.Generic

let d = new Dictionary<int, int>()

for i=1 to 1000000000 do
    d.Add(i,i)

我对 C5 收藏库进行了相同的测试.结果是C5中的字典可能用完整个内存.代码使用C5:

I did a same test to the C5 collection library. The result is that the dictionary in C5 could use up the whole memory. The code uses C5:

let d = C5.HashDictionary<int, int> ()
for i=1 to 1000000000 do
    d.Add(i,i)

有人知道为什么吗?

推荐答案

Microsoft CLR 的最大对象大小限制为 2GB，即使是 64 位版本也是如此.(我不确定这个限制是否也存在于 Mono 等其他实现中.)

The Microsoft CLR has a 2GB maximum object size limit, even the 64 bit version. (I'm not sure whether this limit is also present in other implementations such as Mono.)

该限制适用于每个单个对象——而不是所有对象的总大小——这意味着使用某种复合集合相对容易解决.

The limitation applies to each single object -- not the total size of all objects -- which means that it's relatively easy to workaround using a composite collection of some sort.

这里有一个讨论和一些示例代码...

There's a discussion and some example code here...

• BigArray，绕过 2GB 阵列大小限制

似乎很少有官方文档提到这个限制.毕竟，它只是当前 CLR 的一个实现细节.我所知道的唯一提及是在此页面上:

There seems to be very little official documentation that refers to this limit. It is, after all, just an implementation detail of the current CLR. The only mention that I'm aware of is on this page:

当您运行 64 位托管64 位 Windows 上的应用程序操作系统，您可以创建一个不超过 2 GB 的对象(GB).

When you run a 64-bit managed application on a 64-bit Windows operating system, you can create an object of no more than 2 gigabytes (GB).

这篇关于.Net 中非常大的集合导致内存不足异常的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！