16GB的系统上256MB分配：OutOfMemoryException异常而填充的MemoryStream [英] OutOfMemoryException while populating MemoryStream: 256MB allocation on 16GB system

问题描述

我是一个64位的Windows 7计算机上运行我的开发IIS服务器（从VS2010 IDE）以下方法与安装的RAM和16GB：

I'm running the following method on my development IIS server (from VS2010 IDE) on a 64-bit Windows 7 machine with 16GB of installed RAM:

public static MemoryStream copyStreamIntoMemoryStream(Stream stream)
{
    long uiLen = stream.Length;
    byte[] buff = new byte[0x8000];

    int nSz;
    MemoryStream ms = new MemoryStream();
    try
    {
        while ((nSz = stream.Read(buff, 0, buff.Length)) != 0)
        {
            ms.Write(buff, 0, nSz);
        }
    }
    finally
    {
        Debug.WriteLine("Alloc size=" + ms.Length);
    }

    return ms;
}

和我得到的的System.OutOfMemoryException 在这条线：

ms.Write(buff, 0, nSz);

在268435456字节分配时抛出：

That is thrown when 268435456 bytes are allocated:

的Alloc大小= 268435456

Alloc size=268435456

这是0x10000000处或256 MB。所以我想知道如果有，我需要设置，使其工作的一些全局设置？

which is 0x10000000 or 256 MB. So I'm wondering if there's some global setting that I need to set to make it work?

下面是配置设置为项目的截图：

Here's a screenshot of the configuration setting for the project:

推荐答案

简短的回答 - 开发服务器是32位进程

Short answer - dev server is 32bit process.

长的答案：为什么只256Mb的？

Long answer for "why just 256Mb?"

首先，让我们来了解它是如何工作的。

First of all, let's understand how it works.

的MemoryStream有内部字节[]缓冲区，以保持所有数据。这不可能predict这个缓冲区的确切大小，所以它只是用一些初始值初始化。

MemoryStream has internal byte[] buffer to keep all the data. It cannot predict exact size of this buffer, so it just initializes it with some initial value.

位置和长度属性不反映实际缓冲区大小 - 它们是逻辑值以反映多少字节被写入，并且容易可能比实际的物理缓冲器尺寸较小

Position and Length properties don't reflect actual buffer size - they are logical values to reflect how many bytes is written, and easily may be smaller than actual physical buffer size.

在此内部缓冲区不能适合所有的数据，应该是调整大小，但在现实生活中它的意思的创建新的缓冲的两倍大小，previous之一，然后复制旧缓冲区中的数据到新的缓冲区。

When this internal buffer can not fit all the data, it should be "re-sized", but in real life it means creating new buffer twice as size as previous one, and then copying data from old buffer to new buffer.

所以，如果您的缓冲区的长度为256MB，而你需要写入新的数据，这意味着净需要寻找但另一个数据块512兆 - 已具备了所有的休息，所以堆应该在在内存分配的那一刻起，当您收到内存不足768MB以上。

So, if the length of your buffer is 256Mb, and you need new data to be written, this means that .Net need to find yet another 512Mb block of data - having all the rest in place, so heap should be at least 768Mb on the moment of memory allocation when you receive OutOfMemory.

另外，请注意，在默认情况下没有一个单一的对象，包括数组，在.net中可以采取超过2GB大小。

Also please notice that by default no single object, including arrays, in .Net can take more than 2Gb in size.

好了，所以这里是它模拟发生了什么样件：

Ok, so here is the sample piece which simulates what's happening:

        byte[] buf = new byte[32768 - 10];

        for (; ; )
        {
            long newSize = (long)buf.Length * 2;
            Console.WriteLine(newSize);

            if (newSize > int.MaxValue)
            {
                Console.WriteLine("Now we reach the max 2Gb per single object, stopping");
                break;
            }

            var newbuf = new byte[newSize];
            Array.Copy(buf, newbuf, buf.Length);
            buf = newbuf;
        }

如果它内置64 /值为anycpu从控制台上运行 - 一切正常。

If it built in x64/AnyCPU and runs from console - everything is ok.

如果它横跨86内置 - 它无法在控制台中。

If it built across x86 - it fails in console.

如果你把它说的Page_Load，建于x64和从VS.Net Web服务器开放 - 失败

If you put it to say Page_Load, built in x64, and open from VS.Net web server - it fails.

如果你做同样的IIS - 一切正常。

If you do the same with IIS - everything is ok.

希望这有助于。

这篇关于16GB的系统上256MB分配：OutOfMemoryException异常而填充的MemoryStream的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！