LOH 碎片化 - 2015 更新 [英] LOH fragmentation - 2015 update

问题描述

有很多关于 .NET LOH 的信息，并且已经在多篇文章中进行了解释.不过，好像有些文章不够严谨.

There is a lot of information available about the .NET LOH and it has been explained in various articles. However, it seems that some articles lack a bit of precision.

在 Brian Rasmussen 的回答 (2009) 中，Microsoft 的程序经理，他说限制是 85000 字节.他还让我们知道还有一个更奇怪的 double[] 案例，其大小为 1000 个元素.相同的 85000 限制由 Maoni Stephens(MSDN，2008 年)，CLR 团队.

In Brian Rasmussen's answer (2009), program manager at Microsoft, he says the limit is 85000 bytes. He also let's us know that there is an even more curious case of double[] with a size of 1000 elements. The same 85000 limit is stated by Maoni Stephens (MSDN, 2008), member of the CLR team.

在评论中，Brian Rasmussen 变得更加准确，让我们知道它可以使用 85000 字节 - 12 字节的 byte[] 进行复制.

In the comments, Brian Rasmussen becomes even more exact and let's us know that it can be reproduced with a byte[] of 85000 bytes - 12 bytes.

Mario Hewardt(高级 Windows 调试"的作者) 在 2013 年告诉我们，.NET 4.5.1 现在也可以压缩 LOH，如果我们告诉它这样做的话.由于默认情况下它是关闭的，除非您已经意识到它，否则问题仍然存在.

Mario Hewardt (author of 'Advanced Windows Debugging') told us in 2013 that .NET 4.5.1 can now compact the LOH as well, if we tell it to do so. Since it is turned off by default, the problem remains unless you're aware of it already.

我无法再重现 byte[] 示例.使用简短的蛮力算法，我发现我必须减去 24(SOH 中的 byte[84999-24]，LOH 中的 byte[85000-24]):

I can't reproduce the byte[] example any more. With a short brute-force algorithm, I found out that I have to subtract 24 instead (byte[84999-24] in SOH, byte[85000-24] in LOH):

    static void Main(string[] args)
    {
        int diff = 0;
        int generation = 3;
        while (generation > 0)
        {
            diff++;
            byte[] large = new byte[85000-diff];
            generation = GC.GetGeneration(large);
        }            
        Console.WriteLine(diff);
    }

我也无法重现 double[] 语句.蛮力给了我 10622 个元素作为边界(double[10621] 在 SOH，double[10622] 在 LOH):

I also couldn't reproduce the double[] statement. Brute-forcing gives me 10622 elements as the border (double[10621] in SOH, double[10622] in LOH):

    static void Main(string[] args)
    {
        int size = 85000;
        int step = 85000/2;
        while (step>0)
        {
            double[] d = new double[size];
            int generation = GC.GetGeneration(d);
            size += (generation>0)?-step:step;
            step /= 2;
        }
        Console.WriteLine(size);
    }

即使我为较旧的 .NET 框架编译应用程序，也会发生这种情况.它也不依赖于发布或调试版本.

This happens even if I compile the application for older .NET frameworks. It also does not depend on Release or Debug build.

如何解释这些变化?

推荐答案

byte[]例子中从12到24的变化可以用CPU架构从32位到64位的变化来解释.在为 x64 或 AnyCPU 编译的程序中，.NET 开销从 2*4 字节(4 字节对象头 + 4 字节方法表)增加到 2*8 字节(8 字节对象头 + 8 字节方法表).此外，该数组具有 4 个字节(32 位)与 8 个字节(64 位)的长度属性.

The change from 12 to 24 in the byte[] example can be explained by the change in CPU architecture from 32 to 64 bit. In programs compiled for x64 or AnyCPU, the .NET overhead increases from 2*4 bytes (4 bytes Object Header + 4 bytes Method Table) to 2*8 bytes (8 bytes Object Header + 8 bytes Method Table). In addition, the array has a length property of 4 bytes (32 bit) versus 8 bytes (64 bits).

对于 double[] 示例，只需使用计算器:85000 bytes/64 bit for the double type = 10625 items，这已经很接近了.考虑到 .NET 开销，结果是(85000 字节 - 24 字节)/每个双精度 8 字节 = 10622 双精度.所以不再对 double[] 进行特殊处理.

For the double[] example, just use a calculator: 85000 bytes / 64 bit for the double type = 10625 items, which is already close. Considering the .NET overhead, the result is (85000 bytes - 24 bytes) / 8 bytes per double = 10622 doubles. So there is no special handling of double[] any more.

顺便说一句，我之前从未找到任何 LOH 碎片的工作演示，所以我自己写了一个.只需为 x86 编译以下代码并运行它.它甚至包括一些调试提示.

BTW, I have never found any working demonstration for LOH fragmentation before, so I wrote one myself. Just compile the following code for x86 and run it. It even includes some debugging hints.

编译为 x64 时效果不佳，因为 Windows 可能会增加页面文件的大小，因此后续 20 MB 内存的分配可能会再次成功.

It won't work as well when compiled as x64 since Windows might increase the size of the pagefile, so the subsequent allocation of 20 MB memory could be successful again.

class Program
{
    static IList<byte[]> small = new List<byte[]>();
    static IList<byte[]> big = new List<byte[]>(); 

    static void Main()
    {
        int totalMB = 0;
        try
        {
            Console.WriteLine("Allocating memory...");
            while (true)
            {
                big.Add(new byte[10*1024*1024]);
                small.Add(new byte[85000-3*IntPtr.Size]);
                totalMB += 10;
                Console.WriteLine("{0} MB allocated", totalMB);
            }
        }
        catch (OutOfMemoryException)
        {
            Console.WriteLine("Memory is full now. Attach and debug if you like. Press Enter when done.");
            Console.WriteLine("For WinDbg, try `!address -summary` and  `!dumpheap -stat`.");
            Console.ReadLine();

            big.Clear();
            GC.Collect();
            Console.WriteLine("Lots of memory has been freed. Check again with the same commands.");
            Console.ReadLine();

            try
            {
                big.Add(new byte[20*1024*1024]);
            }
            catch(OutOfMemoryException)
            {
                Console.WriteLine("It was not possible to allocate 20 MB although {0} MB are free.", totalMB);
                Console.ReadLine();
            }
        }
    }
}

这篇关于LOH 碎片化 - 2015 更新的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！