.Net字典< int，int>大约6,000,000个条目的内存异常异常 [英] .Net Dictionary<int,int> out of memory exception at around 6,000,000 entries

问题描述

我正在使用字典< Int，Int> 将颜色的频率存储在图像中，其中键是颜色（以int为单位），以及该值是在图像中找到颜色的次数。

当我处理更大/更丰富多彩的图像时，这个字典增长非常大。我在大约6,000,000个条目中得到了一个内存异常。这是32位模式下运行的预期容量吗？如果是这样，有什么可以做的吗？还有一些替代方法可以跟踪这些不会用尽内存的数据？

为了引用，这里是循环遍历像素的代码一个位图并将频率保存在 Dictionary< int，int> 中：

 位图b; // = something ... 
 Dictionary< int，int> count = new Dictionary< int，int>（）; 
 System.Drawing.Color颜色; （int i = 0; i  
（int j = 0; j< b.Height; j ++）
 
 b $ b {
 color = b.GetPixel（i，j）; 
 int colorString = color.ToArgb（）; 
 if（！count.Keys.Contains（color.ToArgb（）））
 {
 count.Add（colorString，0）; 
} 
 count [colorString] = count [colorString] + 1; 
} 
} 
  

编辑：如果你想知道有什么图像许多不同的颜色： http://allrgb.com/images/mandelbrot.png

编辑：我还应该提到这是在使用.Net 4.0的asp.net Web应用程序中运行。所以可能会有额外的内存限制。

编辑：我刚刚在控制台应用程序中运行相同的代码，没有任何问题。问题只发生在ASP.Net。

解决方案

更新：鉴于OP的示例图像，似乎最大数量的项目将超过1600万，而显然是，即使在实例化字典时也不能分配。我在这里看到三个选项：

• 将图像的大小调整到可管理的大小，并从中工作。

<

• 转到具有其他人建议的固定大小的数组。

以前的答案：问题是您没有为您的字典分配足够的空间。在某些时候，当它正在扩展时，你只是用尽了扩展的内存，但不一定是新的字典。

示例：这个代码用完了内存在近2400万条记录（在我的机器中，以32位模式运行）：

 字典< int，int> count = new Dictionary< int，int>（）; （int i = 0;; i ++）
 count.Add（i，i）; 
 
  

因为最后一次扩展，它目前正在为已经存在的条目使用空格，并尝试分配<如果我们最初分配空间，比如4000万条目，那么现在， ，它运行没有问题：

 字典< int，int> count = new Dictionary< int，int>（40000000）; 
  

所以尝试指出创建字典时将有多少条目。

从 MSDN ：

字典的容量是可以在调整大小之前添加到字典的元素数量。当元素被添加到词典时，根据重新分配内部数组的需要自动增加容量。
如果可以估计集合的大小，指定初始容量将消除在向Dictionary 添加元素时执行大量调整大小操作的需要。

I am using a Dictionary<Int,Int> to store the frequency of colors in an image, where the key is the the color (as an int), and the value is the number of times the color has been found in the image.

When I process larger / more colorful images, this dictionary grows very large. I get an out of memory exception at just around 6,000,000 entries. Is this the expected capacity when running in 32-bit mode? If so, is there anything I can do about it? And what might be some alternative methods of keeping track of this data that won't run out of memory?

For reference, here is the code that loops through the pixels in a bitmap and saves the frequency in the Dictionary<int,int>:

Bitmap b; // = something...
Dictionary<int, int> count = new Dictionary<int, int>();
System.Drawing.Color color;

for (int i = 0; i < b.Width; i++)
{
    for (int j = 0; j < b.Height; j++)
    {
        color = b.GetPixel(i, j);
        int colorString = color.ToArgb();
        if (!count.Keys.Contains(color.ToArgb()))
        {
            count.Add(colorString, 0);                
        }
        count[colorString] = count[colorString] + 1;
    }
}

Edit: In case you were wondering what image has that many different colors in it: http://allrgb.com/images/mandelbrot.png

Edit: I also should mention that this is running inside an asp.net web application using .Net 4.0. So there may be additional memory restrictions.

Edit: I just ran the same code inside a console application and had no problems. The problem only happens in ASP.Net.

解决方案

Update: Given the OP's sample image, it seems that the maximum number of items would be over 16 million, and apparently even that is too much to allocate when instantiating the dictionary. I see three options here:

• Resize the image down to a manageable size and work from that.
• Try to convert to a color scheme with fewer color possibilities.
• Go for an array of fixed size as others have suggested.

Previous answer: the problem is that you don't allocate enough space for your dictionary. At some point, when it is expanding, you just run out of memory for the expansion, but not necessarily for the new dictionary.

Example: this code runs out of memory at nearly 24 million entries (in my machine, running in 32-bit mode):

Dictionary<int, int> count = new Dictionary<int, int>();
for (int i = 0; ; i++)
     count.Add(i, i);

because with the last expansion it is currently using space for the entries already there, and tries to allocate new space for another so many million more, and that is too much.

Now, if we initially allocate space for, say, 40 million entries, it runs without problem:

Dictionary<int, int> count = new Dictionary<int, int>(40000000);

So try to indicate how many entries there will be when creating the dictionary.

From MSDN:

The capacity of a Dictionary is the number of elements that can be added to the Dictionary before resizing is necessary. As elements are added to a Dictionary, the capacity is automatically increased as required by reallocating the internal array. If the size of the collection can be estimated, specifying the initial capacity eliminates the need to perform a number of resizing operations while adding elements to the Dictionary.

这篇关于.Net字典&lt; int，int&gt;大约6,000,000个条目的内存异常异常的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！