你能用 List<List<struct>> 吗?绕过2gb 对象限制? [英] Can you use List&lt;List&lt;struct&gt;&gt; to get around the 2gb object limit?

问题描述

我在 C# 中遇到了 2gb 对象限制(由于某些烦人的原因，这甚至适用于 64 位)和大量结构体(估计总共 4.2 gig).

I'm running up against the 2gb object limit in c# (this applies even in 64 bit for some annoying reason) with a large collection of structs (est. size of 4.2 gig in total).

现在显然使用 List 会给我一个大小为 4.2gb 的列表，但是使用由较小列表组成的列表，这些列表又包含一部分结构，是否允许我跳过这个限制?

Now obviously using List is going to give me a list of size 4.2gb give or take, but would using a list made up of smaller lists, which in turn contain a portion of the structs, allow me to jump this limit?

我的理由是，这只是 CLR 中的一个硬编码限制，它阻止我在 64 位平台上实例化 9gig 对象，并且它与系统资源完全无关.列表和数组也是引用类型，因此包含列表的列表实际上只包含对每个列表的引用.因此没有任何对象超过大小限制.

My reasoning here is that it's only a hard-coded limit in the CLR that stops me instantiating a 9gig object on my 64bit platform, and it's entirely unrelated to system resources. Also Lists and Arrays are reference types, and so a List containing lists would only actually contain the references to each list. No one object therefore exceeds the size limit.

有什么理由不能这样做吗?我现在想自己试试这个，但我手头没有内存分析器来验证.

Is there any reason why this wouldn't work? I'd try this myself right now but I don't have a memory profiler on hand to verify.

推荐答案

现在显然使用 List 会给我一个大小为 4.2gb 的列表，但是使用由较小列表组成的列表，这些列表又包含一部分结构，是否允许我跳过这个限制?

Now obviously using List is going to give me a list of size 4.2gb give or take, but would using a list made up of smaller lists, which in turn contain a portion of the structs, allow me to jump this limit?

是的 - 不过，如果您想解决这个限制，我会考虑自己使用数组，而不是让 List 类管理数组.

Yes - though, if you're trying to work around this limit, I'd consider using arrays yourself instead of letting the List<T> class manage the array.

CLR 中的 2gb 单个对象限制就是单个对象实例.当您创建结构数组(List 在内部使用)时，整个数组是 CLR 中的一个对象实例".但是，通过使用 List> 或锯齿状数组，每个内部列表/数组都是一个单独的对象，它允许您有效地拥有您希望的任何大小的对象.

The 2gb single object limit in the CLR is exactly that, a single object instance. When you make an array of a struct (which List<T> uses internally), the entire array is "one object instance" in the CLR. However, by using a List<List<T>> or a jagged array, each internal list/array is a separate object, which allows you to effectively have any size object you wish.

CLR 团队实际上写了关于此的博客，并提供了一个示例<代码>BigArray<T> 实现就像单个 List，但在内部为您进行块"管理.这是获取 >2gb 列表的另一种选择.

The CLR team actually blogged about this, and provided a sample BigArray<T> implementation that acts like a single List<T>, but does the "block" management internally for you. This is another option for getting >2gb lists.

请注意，.NET 4.5 可以选择提供 大于x64 上的 2gb 对象，但这将是您必须明确选择拥有的东西.

Note that .NET 4.5 will have the option to provide larger than 2gb objects on x64, but it will be something you have to explicitly opt in to having.

这篇关于你能用 List<List<struct>> 吗?绕过2gb 对象限制?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！