了解内存性能计数器 [英] Understanding Memory Performance Counters

问题描述

[更新-2010年9月30日]

[Update - Sep 30, 2010]

由于我对此进行了很多研究&相关主题，我将写下我从经验中得到的所有技巧以及在此处的答案中提供的建议-

Since I studied a lot on this & related topics, I'll write whatever tips I gathered out of my experiences and suggestions provided in answers over here-

1)使用内存事件探查器(首先尝试CLR事件探查器)，查找消耗最大内存的例程并对其进行微调，例如重用大数组，并尝试将对对象的引用保持在最低限度.

1) Use memory profiler (try CLR Profiler, to start with) and find the routines which consume max mem and fine tune them, like reuse big arrays, try to keep references to objects to minimal.

2)如果可能，请分配小对象(对于.NET 2.0，应分配小于85k的对象)，并使用内存池(如果可以避免垃圾收集器占用大量CPU).

2) If possible, allocate small objects (less than 85k for .NET 2.0) and use memory pools if you can to avoid high CPU usage by garbage collector.

3)如果增加了对对象的引用，则有责任将它们取消引用的次数相同.您将放心，代码可能会更好.

3) If you increase references to objects, you're responsible to de-reference them the same number of times. You'll have peace of mind and code probably will work better.

4)如果仍然无法解决问题，请使用消除方法(注释/跳过代码)来找出消耗最大内存的内存.

4) If nothing works and you are still clueless, use elimination method (comment/skip code) to find out what is consuming most memory.

在代码中使用内存性能计数器也可能会有所帮助.

Using memory performance counters inside your code might also help you.

希望这些帮助！

[原始问题]

嗨！

我正在使用C#，但是我的问题是内存不足异常.

I'm working in C#, and my issue is out of memory exception.

我在这里阅读了一篇有关LOH的出色文章-> http://www.simple-talk.com/dotnet/.net-framework/the-dangers-of-the-large-object-heap/

I read an excellent article on LOH here -> http://www.simple-talk.com/dotnet/.net-framework/the-dangers-of-the-large-object-heap/

太棒了！

然后 http://dotnetdebug.ne​​t/2005/06/30/perfmon-your-debugging-buddy/

我的问题:

My issue:

我正面临企业级桌面应用程序中的内存不足问题.我试图阅读和理解有关内存分析和性能计数器的知识(也尝试过WinDBG！-有点)，但是对基本知识还是一无所知.

I am facing out of memory issue in an enterprise level desktop application. I tried to read and understand stuff about memory profiling and performance counter (tried WinDBG also! - little bit) but am still clueless about basic stuff.

我尝试了CLR事件探查器来分析内存使用情况.在以下方面很有帮助:

I tried CLR profiler to analyze the memory usage. It was helpful in:

1. 告诉我谁分配了大块内存

1. Showing me who allocated huge chunks of memory

哪种数据类型占用了最大内存

What data type used maximum memory

但是，CLR Profiler和性能计数器(因为它们共享相同的数据)都无法解释:

But, both, CLR Profiler and Performance Counters (since they share same data), failed to explain:

1. 每次运行该应用程序后收集的数字-如何了解是否有改进?！?！

1. The numbers that is collected after each run of the app - how to understand if there is any improvement?!?!

如何在每次运行后比较性能数据-特定计数器的低/高数是好是坏?

How do I compare the performance data after each run - is lower/higher number of a particular counter good or bad?

---

我需要什么:

我正在寻找有关以下方面的提示:

I am looking for the tips on:

1. 如何释放(是的，正确的)托管的数据类型对象(如数组，大字符串)-但如果可能的话，不进行GC.Collect调用.我必须时不时地处理长度为500KB(不可避免的大小:-())的字节数组.

1. How to free (yes, right) managed data type objects (like arrays, big strings) - but not by making GC.Collect calls, if possible. I have to handle arrays of bytes of length like 500KB (unavoidable size :-( ) every now and then.

如果发生碎片，如何压缩内存-似乎.NET GC并没有真正有效地执行该操作并导致OOM.

If fragmentation occurs, how to compact memory - as it seems that .NET GC is not really effectively doing that and causing OOM.

此外，LOH的85KB限制到底是多少?这是对象的大小吗?这对我来说不是很清楚.

Also, what exactly is 85KB limit for LOH? Is this the size of the object of the overall size of the array? This is not very clear to me.

哪些内存计数器可以告诉您代码更改是否实际上减少了OOM的机会?

What memory counters can tell if code changes are actually reducing the chances of OOM?

我已经知道的提示

1. 将托管对象设置为null-将它们标记为垃圾-以便垃圾收集器可以收集它们.这很奇怪-将 string [] 对象设置为null后，所有堆中的#个字节都会飙升！

1. Set managed objects to null - mark them garbage - so that garbage collector can collect them. This is strange - after setting a string[] object to null, the # bytes in all Heaps shot up!

避免创建大于85KB的对象/数组-这不在我的控制范围内.因此，可能会有很多LOH.

Avoid creating objects/arrays > 85KB - this is not in my control. So, there could be lots of LOH.

3.

Memory Leaks Indicators:

# bytes in all Heaps increasing
Gen 2 Heap Size increasing
# GC handles increasing
# of Pinned Objects increasing
# total committed Bytes increasing
# total reserved Bytes increasing
Large Object Heap increasing

---

我的情况:

---

My situation:

• 我有4 GB的32位计算机，上面装有Wink 2K3服务器SP2.
• 我了解应用程序可以使用< = 2 GB的物理RAM
• 增加虚拟内存(页面文件)的大小在这种情况下不起作用.

作为其OOM问题，我仅关注与内存相关的计数器.

As its OOM issue, I am only focusing on memory related counters only.

请指教！ 由于缺少好的文档，我真的需要一些帮助！

Please advice! I really need some help as I'm stuck because of lack of good documentation!

推荐答案

Nayan，这是您的问题的答案以及一些其他建议.

Nayan, here are the answers to your questions, and a couple of additional advices.

1. 您无法释放它们，只能使它们更易于被GC收集.看来您已经知道了:关键是减少对该对象的引用数量.
2. 碎片是您无法控制的另一件事.但是有几个因素可能会影响到这一点:
   • LOH外部碎片的危险性不如Gen2外部碎片，因为LOH未被压缩.可以重新使用LOH的空闲插槽.
   • 如果所引用的500Kb字节数组用作某些IO缓冲区(例如传递给某些基于套接字的API或非托管代码)，则很有可能会被固定.固定对象无法通过GC压缩，这是堆碎片最常见的原因之一.
   • 85K是对象大小的限制.但是请记住，System.Array实例也是一个对象，因此您所有的500K byte []都在LOH中.
   • 帖子中的所有计数器都可以提示内存消耗的变化，但是在您的情况下，我将选择BIAH(所有堆中的字节)和LOH大小作为主要指标. BIAH显示所有托管堆的总大小(准确地说是Gen1 + Gen2 + LOH，没有Gen0-但谁在乎Gen0，对吗?:))，而LOH是所有大字节[]所在的堆. /li>

1. You cannot free them, you can only make them easier to be collected by GC. Seems you already know the way:the key is reducing the number of references to the object.
2. Fragmentation is one more thing which you cannot control. But there are several factors which can influence this:
   • LOH external fragmentation is less dangerous than Gen2 external fragmentation, 'cause LOH is not compacted. The free slots of LOH can be reused instead.
   • If the 500Kb byte arrays are referring to are used as some IO buffers (e.g. passed to some socket-based API or unmanaged code), there are high chances that they will get pinned. A pinned object cannot be compacted by GC, and they are one of the most frequent reasons of heap fragmentation.
   • 85K is a limit for an object size. But remember, System.Array instance is an object too, so all your 500K byte[] are in LOH.
   • All counters that are in your post can give a hint about changes in memory consumption, but in your case I would select BIAH (Bytes in all heaps) and LOH size as primary indicators. BIAH show the total size of all managed heaps (Gen1 + Gen2 + LOH, to be precise, no Gen0 - but who cares about Gen0, right? :) ), and LOH is the heap where all large byte[] are placed.

建议:

• 已经提出的一些东西:预先分配和缓冲您的缓冲区.

• Something that already has been proposed: pre-allocate and pool your buffers.

如果可以使用任何集合而不是连续的字节数组(如果在IO中使用缓冲区，则不是这种情况)，这是一种有效的不同方法:实现一个内部将由以下组成的自定义集合:许多较小的数组.这类似于C ++ STL库中的std :: deque.由于每个单独的数组都小于85K，因此整个集合不会进入LOH.使用此方法可获得的优势如下:仅在发生完整GC时才收集LOH.如果您的应用程序中的byte []寿命不是很长，并且(如果它们的大小较小)在被收集之前会进入Gen0或Gen1，则这将使GC的内存管理变得更加容易，因为Gen2收集的重量更重.

A different approach which can be effective if you can use any collection instead of contigous array of bytes (this is not the case if the buffers are used in IO): implement a custom collection which internally will be composed of many smaller-sized arrays. This is something similar to std::deque from C++ STL library. Since each individual array will be smaller than 85K, the whole collection won't get in LOH. The advantage you can get with this approach is the following: LOH is only collected when a full GC happens. If the byte[] in your application are not long-lived, and (if they were smaller in size) would get in Gen0 or Gen1 before being collected, this would make memory management for GC much easier, since Gen2 collection is much more heavyweight.

有关测试和测试的建议监视方法:根据我的经验，需要长时间监视GC行为，内存占用量以及其他与内存相关的内容，以获取一些有效且稳定的数据.因此，每次更改代码中的某些内容时，都要进行足够长的测试，以监视内存性能计数器以查看更改的影响.

An advice on the testing & monitoring approach: in my experience, the GC behavior, memory footprint and other memory-related stuff need to be monitored for quite a long time to get some valid and stable data. So each time you change something in the code, have a long enough test with monitoring the memory performance counters to see the impact of the change.

我还建议您查看GC计数器中的％Time，因为它可以很好地指示内存管理的有效性.该值越大，您的应用程序花费在GC例程上的时间就越多，而不是花费在处理用户的请求或执行其他有用的"操作上.对于该计数器的绝对值指示问题，我无法给出建议，但是我可以分享经验，供您参考:对于我正在处理的应用程序，我们通常将GC中的％Time视为20％以上. p>

I would also recommend to take a look at % Time in GC counter, as it can be a good indicator of the effectiveness of memory management. The larger this value is, the more time your application spends on GC routines instead of processing the requests from users or doing other 'useful' operations. I cannot give advices for what absolute values of this counter indicate an issue, but I can share my experience for your reference: for the application I am working on, we usually treat % Time in GC higher than 20% as an issue.

此外，如果您共享应用程序中与内存相关的性能计数器的某些值，这将非常有用:专用字节和进程的工作集，BIAH，已提交的总字节数，LOH大小，Gen0，Gen1，Gen2大小，# Gen0，Gen1，Gen2集合数，％GC时间.这将有助于更好地了解您的问题.

Also, it would be useful if you shared some values of memory-related perf counters of your application: Private bytes and Working set of the process, BIAH, Total committed bytes, LOH size, Gen0, Gen1, Gen2 size, # of Gen0, Gen1, Gen2 collections, % Time in GC. This would help better understand your issue.

这篇关于了解内存性能计数器的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！