我怎样才能prevent BufferManager / PooledBufferManager在我的WCF客户端应用程序从浪费内存？ [英] How can I prevent BufferManager / PooledBufferManager in my WCF client app from wasting memory?
问题描述
分析WCF客户端应用程序（我没有写，还是不知道的太多），它通过SOAP和运行几天就会抛出一个OutOfMemoryException后谈了一堆的服务，我发现。净的PooledBufferManager永远不会释放未使用的缓冲区，即使在应用程序运行的内存，从而导致OOMEs。
这当然符合该规范是：<一href="http://msdn.microsoft.com/en-us/library/ms405814.aspx">http://msdn.microsoft.com/en-us/library/ms405814.aspx
池和缓冲区[...]毁坏了，当缓冲池 通过垃圾回收。
请随意回答下面的问题只有一个，因为我有一大堆的问题，一些更具普遍性，有的具体到我们的应用程序的使用BufferManager的。
首先一个关于（默认池）BufferManager几个常规问题：
1）在这里我们有GC一个环境，为什么我们需要一个BufferManager将保存到未使用的内存，即使是导致OOME？我知道，有BufferManager.Clear（），你可以用它来的手动的摆脱掉所有的缓冲区 - 如果你有机会到BufferManager，那是。为什么我似乎没有能够获得见进一步下跌。
2）尽管MS索赔的说：这个过程比创建每次你需要使用一次性摧毁一个缓冲快多了。，不应该离开，截至到GC （和它的LOH为例）和优化GC呢？
3）当执行BufferManager.Take（33 * 1024 * 1024），我会得到64M的缓存，因为PooledBufferManager将缓存缓冲区以备后用，这可能 - 好吧，在我的情况下，它不是，因此它的内存纯粹是浪费 - 是的，也就是说，34M，50M或或64M，是必要的。因此，它是明智的创建一个潜在的非常浪费BufferManager这样，所使用（默认情况下，我认为）由HttpsChannelFactory？我没有看到为内存分配的性能怎么样应该的问题，特别是当我们谈论的是WCF和网络服务的应用程序将要每10秒上衣，通常有更多秒钟甚至几分钟。
现在一些关系到我们的应用程序使用BufferManagers的更具体的问题。该应用程序连接到几个不同的WCF服务。对于每个人，我们维持对HTTP连接的连接池，因为可能同时发生的连接。
检查在一个堆转储，这仅被使用过一次在我们的应用程序在初始化时，不事后需要64M字节数组的一个最大的对象，从服务的响应是，大的只有在初始化时，这顺便说一句。是典型的许多应用程序我都用过，尽管这可能会受到opimization（缓存到磁盘等）。在WinDbg中采用GC根分析，得到以下（我消毒，我们的专利类的名称为MyServiceX，等等。）：
0：000：86＆GT; ！gcroot -nostacks 193e1000
域（00B8CCD0）：手柄（已固定）：4d1330：根：0e5b9c50（System.Object的[]） - ＆GT;
035064f0（MyServiceManager） - ＆GT;
0382191c（MyHttpConnectionPool`1 [[MyServiceX，MyLib中]]） - ＆GT;
03821988（System.Collections.Generic.Queue`1 [[MyServiceX，MyLib中]]） - ＆GT;
038219a8（System.Object的[]） - ＆GT;
039c05b4（System.Runtime.Remoting.Proxies .__ TransparentProxy） - ＆GT;
039c0578（System.ServiceModel.Channels.ServiceChannelProxy） - ＆GT;
039c0494（System.ServiceModel.Channels.ServiceChannel） - ＆GT;
039bee30（System.ServiceModel.Channels.ServiceChannelFactory + ServiceChannelFactoryOverRequest） - ＆GT;
039beea4（System.ServiceModel.Channels.HttpsChannelFactory） - ＆GT;
039bf2c0（System.ServiceModel.Channels.BufferManager + PooledBufferManager） - ＆GT;
039c02f4（System.Object的[]） - ＆GT;
039bff24（System.ServiceModel.Channels.BufferManager + PooledBufferManager +缓冲池） - ＆GT;
039bff44（System.ServiceModel.SynchronizedPool`1 [System.Byte []，mscorlib程序]） - ＆GT;
039bffa0（System.ServiceModel.SynchronizedPool`1 + GlobalPool [System.Byte []，mscorlib程序]） - ＆GT;
039bffb0（System.Collections.Generic.Stack`1 [System.Byte []，mscorlib程序]） - ＆GT;
12bda2bc（System.Byte [] []） - ＆GT;
193e1000（System.Byte []）
看着GC根由一个BufferManager管理的其他字节数组显示，其他服务（不'MyServiceX）有不同的缓冲情况下，让每个人是在浪费自己的记忆，他们甚至不共享的浪费。
4）我们在这里做什么事吗？我不是一个WCF专家以任何方式，所以我们可以做出各种HttpsChannelFactory实例都使用相同的BufferManager？
5）或甚至更好，可能我们只是告诉所有HttpsChannelFactory情况下不要使用BufferManagers所有，并要求GC做的神该死的工作，这是'管理内存<？ / P>
6）如果问题4）和5）不能回答，我能获得所有HttpsChannelFactory实例的BufferManager和手动调用.Clear（）他们 - 这是远远上最佳的解决方案，但它已经帮助，在我的情况下，它会释放不仅前述方式64M，64M，但32M + 16M + + 8M + 4M + 2M只是一个服务实例！因此，仅将让我的应用程序持续更长的时间，但不运行到内存的问题（没有，我们没有内存泄漏问题，比BufferManager等，虽然我们消耗了大量的内存和在过程中积累了大量的数据很多天，但是这不是这里的问题）
4）难道我们做错了什么吗？我不是一个WCF专家通过任何 手段，所以我们可以做出各种HttpsChannelFactory实例的所有 使用同一个BufferManager？
5），或者甚至更好，可能我们只是告诉所有HttpsChannelFactory 实例不要使用BufferManagers所有，并要求GC做的 神该死的工作，这是管理存储？
我想解决这些问题2，可以改变从'缓冲'到'流'的transferMode的一种方式。则要进行调查，为流模式有几个限制，我可能无法使用它。
更新：它实际上的伟大工程！我的内存消耗在缓冲模式下的应用程序的启动过程中的 630M 在高峰时间，并减少为 470M ：当满载。切换到流模式后，内存消耗不显示临时峰值和满载时，消费是唯一的 270M ！
顺便说一句，这是在客户端应用程序code对我来说是单行的变化。我刚加入这一行：
httpsTransportBindingElement.TransferMode = TransferMode.StreamedResponse;
Analyzing a WCF client application (that I did not write and still do not know too much about) that talks to a bunch of services via SOAP and after running for a couple of days will throw an OutOfMemoryException, I found out that .net's PooledBufferManager will never ever release unused buffers, even when the app runs out of memory, leading to OOMEs.
This of course being in accordance with the spec: http://msdn.microsoft.com/en-us/library/ms405814.aspx
The pool and its buffers are [...] destroyed when the buffer pool is reclaimed by garbage collection.
Please feel free to answer to only a single of the questions below, as I have a bunch of questions, some of a more general nature, and some specific to our app's use of the BufferManager.
First a couple of general questions about the (default Pooled)BufferManager:
1) In a environment where we have GC, why would we need a BufferManager that will hold on to unused memory, even when that leads to OOME? I know, there is BufferManager.Clear(), which you can use to manually get rid off all buffers - if you have access to the BufferManager, that is. See further down for why I don't seem to have access.
2) Despite of MS' claim that "This process is much faster than creating and destroying a buffer every time you need to use one.", shouldn't they leave that up to the GC (and its LOH for example) and optimize the GC instead?
3) When doing a BufferManager.Take(33 * 1024 * 1024), I will get a buffer of 64M, as the PooledBufferManager will cache that buffer for later reuse, which might - well, in my case it isn't and therefore it's pure waste of memory - be that, say, 34M, or 50M, or 64M, are needed. So was it wise to create a potentially very wasteful BufferManager like this, that is used (by default, I assume) by HttpsChannelFactory? I'm failing to see how the performance for memory allocation should matter, especially when we are talking about WCF and network services that the application will talk to every 10 seconds TOPS, normally many more seconds or even minutes.
Now some more specific questions related to our application's use of BufferManagers. The app connects to a couple of different WCF services. For each of them we maintain a connection pool for the http connections, as connections may occur concurrently.
Inspecting the single biggest object in one heap dump, a 64M byte array that had only been used once in our app at initialization time and is not needed afterwards, as the response from the service is that big only at initialization time, which btw. is typical for many applications I have used, even though that could be subject to opimization (caching to disk etc.). A GC root analysis in WinDbg yields the following (I sanitized the names of our proprietary classes to 'MyServiceX', etc.):
0:000:x86> !gcroot -nostacks 193e1000
DOMAIN(00B8CCD0):HANDLE(Pinned):4d1330:Root:0e5b9c50(System.Object[])->
035064f0(MyServiceManager)->
0382191c(MyHttpConnectionPool`1[[MyServiceX, MyLib]])->
03821988(System.Collections.Generic.Queue`1[[MyServiceX, MyLib]])->
038219a8(System.Object[])->
039c05b4(System.Runtime.Remoting.Proxies.__TransparentProxy)->
039c0578(System.ServiceModel.Channels.ServiceChannelProxy)->
039c0494(System.ServiceModel.Channels.ServiceChannel)->
039bee30(System.ServiceModel.Channels.ServiceChannelFactory+ServiceChannelFactoryOverRequest)->
039beea4(System.ServiceModel.Channels.HttpsChannelFactory)->
039bf2c0(System.ServiceModel.Channels.BufferManager+PooledBufferManager)->
039c02f4(System.Object[])->
039bff24(System.ServiceModel.Channels.BufferManager+PooledBufferManager+BufferPool)->
039bff44(System.ServiceModel.SynchronizedPool`1[[System.Byte[], mscorlib]])->
039bffa0(System.ServiceModel.SynchronizedPool`1+GlobalPool[[System.Byte[], mscorlib]])->
039bffb0(System.Collections.Generic.Stack`1[[System.Byte[], mscorlib]])->
12bda2bc(System.Byte[][])->
193e1000(System.Byte[])
Looking at gc roots for other byte arrays managed by a BufferManager reveals that other services (not 'MyServiceX') have different BufferPool instances, so each one is wasting their own memory, they are not even sharing the waste.
4) Are we doing something wrong here? I'm not a WCF expert by any means, so could we make the various HttpsChannelFactory instances all use the same BufferManager?
5) Or maybe even better, could we just tell all HttpsChannelFactory instances NOT to use BufferManagers at all and ask the GC to do its god-damn job, which is 'managing memory'?
6) If questions 4) and 5) can't be answered, could I get access to the BufferManager of all HttpsChannelFactory instances and manually call .Clear() on them - this is far from on optimal solution, but it would already help, in my case, it would free not only the aformentioned 64M, but 64M + 32M + 16M + 8M + 4M + 2M just in one service instance! So that alone would make my app last much longer without running into memory problems (and no, we don't have a memory leak issue, other than BufferManager, although we do consume a lot of memory and accumulate a lot of data over the course of many days, but that's not the issue here)
4) Are we doing something wrong here? I'm not a WCF expert by any means, so could we make the various HttpsChannelFactory instances all use the same BufferManager?
5) Or maybe even better, could we just tell all HttpsChannelFactory instances NOT to use BufferManagers at all and ask the GC to do its god-damn job, which is 'managing memory'?
I guess one way of addressing those 2 questions could be changing the TransferMode from 'buffered' to 'streamed'. Will have to investigate, as 'streamed' mode has a couple of limitations and I might not be able to use it.
Update: It actually works great! My memory consumption in buffered mode during startup of the app was 630M at peak times, and reduced to 470M when fully loaded. After switching to streamed mode, memory consumption does not show a temporary peak and when fully loaded, consumption is at only 270M!
Btw., this was a one-line change in the client app code for me. I just had to add this line:
httpsTransportBindingElement.TransferMode = TransferMode.StreamedResponse;
这篇关于我怎样才能prevent BufferManager / PooledBufferManager在我的WCF客户端应用程序从浪费内存？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
