.NET 可用内存使用情况(如何防止过度分配/释放内存给操作系统) [英] .NET Free memory usage (how to prevent overallocation / release memory to the OS)

问题描述

我目前正在开发一个网站，该网站大量使用缓存数据来避免往返.在启动时，我们得到一个大"图(数百个不同种类的对象).这些对象通过 WCF 检索并反序列化(我们使用协议缓冲区进行序列化)我正在使用 redgate 的内存分析器来调试内存问题(内存似乎不适合我们在"完成初始化并最终得到此报告后需要多少内存

I'm currently working on a website that makes large use of cached data to avoid roundtrips. At startup we get a "large" graph (hundreds of thouthands of different kinds of objects). Those objects are retrieved over WCF and deserialized (we use protocol buffers for serialization) I'm using redgate's memory profiler to debug memory issues (the memory didn't seem to fit with how much memory we should need "after" we're done initializing and end up with this report

现在我们可以从这份报告中收集到的是:

Now what we can gather from this report is that:

1) .NET 分配的大部分内存都是免费的(它可能在反序列化期间被正确分配，但现在它是免费的，我希望它返回给操作系统)

1) Most of the memory .NET allocated is free (it may have been rightfully allocated during deserialisation, but now that it's free, i'd like for it to return to the OS)

2) 内存是碎片化的(这很糟糕，因为每次我刷新现金时，我都需要重做需要大量内存的反序列化过程，这反过来又会创建可能由于碎片化而引发 OutOfMemoryException 的大对象)

2) Memory is fragmented (which is bad, as everytime i refresh the cash i need to redo the memory hungry deserialisation process and this, in turn creates large object that may throw an OutOfMemoryException due to fragmentation)

3) 我不知道为什么空间是碎片的，因为当我查看大对象堆时，只有 30 个实例，15 个 object[] 直接附加到 GC 并且与我完全无关，1 是char 数组也直接附加到 GC 堆，其余 15 个是我的，但不是原因，因为如果我在代码中将它们注释掉，我会得到相同的报告.

3) I have no clue why the space is fragmented, because when i look at the large object heap, there are only 30 instances, 15 object[] are directly attached to the GC and totally unrelated to me, 1 is a char array also attached directly to the GC Heap, the remaining 15 are mine but are not the cause of this as i get the same report if i comment them out in code.

所以我的问题是，我该怎么做才能更进一步?我不太确定在调试/工具中要寻找什么，因为我的内存似乎是碎片化的，但不是我自己，而且 .net 分配了大量可用空间，我无法释放.

So my question is, what can i do to go further with this? I'm not really sure what to look for in debugging / tools as it seems my memory is fragmented, but not by me, and huge amounts of free spaces are allocated by .net , which i can't release.

另外，请确保您在回答之前充分理解问题，我不是在寻找一种方法来释放 .net (GC.Collect) 中的内存，而是将 .net 中已经可用的内存释放到系统以及对所述内存进行碎片整理.

Also please make sure you understand the question well before answering, i'm not looking for a way to free memory within .net (GC.Collect), but to free memory that is already free in .net , to the system as well as to defragment said memory.

请注意，缓慢的解决方案很好，如果可以手动对大堆进行碎片整理，我会全力以赴，因为我可以在 RefreshCache 结束时调用它，如果运行需要 1 或 2 秒也没关系.

Note that a slow solution is fine, if it's possible to manually defragment the large heap i'd be all for it as i can call it at the end of RefreshCache and it's ok if it takes 1 or 2 second to run.

感谢您的帮助！

我忘记了一些注意事项:1) 该项目是一个 .net 2.0 网站，我在 .net 4 池中运行它得到相同的结果，如果我在 .net 4 池中运行它并将其转换为 .net 4 并重新编译，则同上.

A few notes i forgot: 1) The project is a .net 2.0 website, i get the same results running it in a .net 4 pool, idem if i run it in a .net 4 pool and convert it to .net 4 and recompile.

2) 这些是发布版本的结果，因此调试版本不是问题.

2) These are results of a release build, so debug build can not be the issue.

3) 这可能非常重要，我在 webdev 服务器中根本没有遇到这些问题，只有在 IIS 中，在 webdev 中我得到的内存消耗与我的实际消耗相当接近(更多，但不是 5-10 倍以上！)

3) And this is probably quite important, i do not get these issues at all in the webdev server, only in IIS, in the webdev i get memory consumption rather close to my actual consumption (well more, but not 5-10X more!)

推荐答案

在大对象堆上分配的对象(对象 >= 85,000 字节，通常是数组)不会被垃圾收集器压缩.微软认为移动这些对象的成本太高了.

Objects allocated on the large object heap (objects >= 85,000 bytes, normally arrays) are not compacted by the garbage collector. Microsoft decided that the cost of moving those objects around would be too high.

建议尽可能重用大对象以避免托管堆和 VM 空间上的碎片.

The recommendation is to reuse large objects if possible to avoid fragmentation on the managed heap and the VM space.

http://msdn.microsoft.com/en-us/magazine/cc534993.aspx

我假设您的大对象是由您的反序列化库创建的临时字节数组.如果库允许您提供自己的字节数组，您可以在程序开始时预分配它们，然后重用它们.

I'm assuming that your large objects are temporary byte arrays created by your deserialization library. If the library allows you to supply your own byte arrays, you could preallocate them at the start of the program and then reuse them.

这篇关于.NET 可用内存使用情况(如何防止过度分配/释放内存给操作系统)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！