的MemoryCache不服从配置的内存限制 [英] MemoryCache does not obey memory limits in configuration
问题描述
我正在使用的.NET 4.0 <一href="http://msdn.microsoft.com/en-us/library/system.runtime.caching.memorycache(v=vs.110).aspx">MemoryCache类在应用程序，并试图限制最大缓存大小，但在我的测试中，它不会出现高速缓存实际上是服从限制。
我使用的设置当中，<一个href="http://msdn.microsoft.com/en-us/library/system.runtime.caching.configuration.memorycacheelement(v=vs.110).aspx">according到MSDN ，都应该限制缓存大小：
- <一个href="http://msdn.microsoft.com/en-us/library/system.runtime.caching.configuration.memorycacheelement.cachememorylimitmegabytes(v=vs.110).aspx">CacheMemoryLimitMegabytes:最大内存大小，以MB为单位，一个对象的实例可以长到。
- <一个href="http://msdn.microsoft.com/en-us/library/system.runtime.caching.configuration.memorycacheelement.physicalmemorylimitpercentage(v=vs.110).aspx">PhysicalMemoryLimitPercentage: 物理内存高速缓存可以使用的比例，pssed作为整数值从1到100 EX $ P $默认值是零，这表明的MemoryCache 实例管理自己的内存 1 的基于存储器被安装在计算机上的量。 1 这并不完全correct--低于4的任何值被忽略，有4所取代。
据我所知，这些值是近似的，而不是硬性限制作为清除缓存被激发每x秒，也取决于轮询间隔和其他无证变量的线程。然而，即使考虑到这些差异，我看到疯狂地不一致的缓存大小时，第一个项目正在从缓存中清除设置后的 CacheMemoryLimitMegabytes 和 PhysicalMemoryLimitPercentage 一起或单独在一个测试应用程序。可以肯定，我跑每个测试10次，计算出的平均数字。
这些是测试下面的例子中code的32位Windows 7 PC的RAM 3GB的结果。缓存大小对每个测试的第一次调用 CacheItemRemoved（）后服用。 （我知道高速缓存的实际尺寸会比这个大）
在第一个到期 MemLimitMB MemLimitPct AVG缓存MB
1 NA 84
2 NA 84
3 NA 84
6 NA 84
NA 1 84
NA 4 84
NA 10 84
10 20 81
10 30 81
10 39 82
10 40 79
10 49 146
10 50 152
10 60 212
10 70 332
10 80 429
10 100 535
100 39 81
500 39 79
900 39 83
1900 39 84
900 41 81
900 46 84
900 49 1.8 GB左右。在任务管理器没有纪念品errros
200 49 156
100 49 153
2000年第60 214
5 60 78
6 60 76
7 100 82
10 100 541
下面是测试应用程序：
使用系统;
使用System.Collections.Generic;
使用System.Collections.Specialized;
使用System.Linq的;
使用System.Runtime.Caching;
使用System.Text;
命名空间FinalCacheTest
{
内部类缓存
{
私有对象Statlock =新的对象（）;
私人诠释ItemCount中;
专用长大小;
私人的MemoryCache memcache的;
私人CacheItemPolicy CIPOL =新CacheItemPolicy（）;
公共缓存（长的CacheSize）
{
CIPOL.RemovedCallback =新CacheEntryRemovedCallback（CacheItemRemoved）;
NameValueCollection中CacheSettings =新的NameValueCollection（3）;
CacheSettings.Add（CacheMemoryLimitMegabytes，Convert.ToString（CacheSize的））;
CacheSettings.Add（physicalMemoryLimitPercentage，Convert.ToString（49））; //此处设置％
CacheSettings.Add（在PollingInterval，Convert.ToString（00:00:10开始））;
MEMCACHE =新的MemoryCache（TestCache，CacheSettings）;
}
公共无效的AddItem（字符串名称，字符串值）
{
CacheItem CI =新CacheItem（名称，值）;
MemCache.Add（CI，CIPOL）;
锁定（Statlock）
{
ItemCount中++;
大小= +（Name.Length + Value.Length * 2）;
}
}
公共无效CacheItemRemoved（CacheEntryRemovedArguments参数数量）
{
Console.WriteLine（缓存包含{0}项目大小为{1}字节，ItemCount中，大小）;
锁定（Statlock）
{
ItemCount--;
大小= - 108;
}
Console.ReadKey（）;
}
}
}
命名空间FinalCacheTest
{
内部类节目
{
私有静态无效的主要（字串[] args）
{
INT MaxAdds = 5000000;
缓存MyCache =新的高速缓存（1）; //设置CacheMemoryLimitMegabytes
的for（int i = 0; I＆LT; MaxAdds;我++）
{
MyCache.AddItem（Guid.NewGuid（）的ToString（），Guid.NewGuid（）的ToString（））;
}
Console.WriteLine（将项目添加到缓存）;
}
}
}
为什么的MemoryCache 不服从所配置的内存限制？
哇，所以我只花了太多的时间与反射的CLR周围挖，但我想我终于等来了什么是怎么回事了良好的手感。
被读取正确的设置，但似乎是在CLR本身看起来像它会使内存限制设置基本上是无用的一个深层次的问题。
下面code为反射出System.Runtime.Caching的DLL，对于CacheMemoryMonitor类（有用于监视物理存储器，并处理其他设置类似的类，但是这是更重要的一个）
保护覆盖INT GetCurrent pressure（）
{
INT NUM = GC.CollectionCount（2）;
SREF REF2 = this._sizedRef;
如果（（NUM = this._gen2Count）及！＆安培;！（REF2 = NULL））
{
this._gen2Count = NUM;
this._idx ^ = 1;
this._cacheSizeSampleTimes [this._idx] = DateTime.UtcNow;
this._cacheSizeSamples [this._idx] = ref2.ApproximateSize;
IMemoryCacheManager经理= s_memoryCacheManager;
如果（经理！= NULL）
{
manager.UpdateCacheSize（this._cacheSizeSamples [this._idx]，this._memoryCache）;
}
}
如果（this._memoryLimit＆LT; = 0L）
{
返回0;
}
长NUM2 = this._cacheSizeSamples [this._idx]
如果（NUM2＆GT; this._memoryLimit）
{
NUM2 = this._memoryLimit;
}
返程（INT）（（NUM2 * 100L）/ this._memoryLimit）;
}
您可能会注意到的第一件事情是，它甚至不尝试一下缓存的大小，直到第二代垃圾收集后，而不是仅仅回落在cacheSizeSamples现有的存储大小值。所以，你永远不会能击中目标的权利，但如果剩下的工作，我们就至少得拿到一个尺寸测量之前，我们真正的麻烦了。
因此，假如一个第二代GC已经发生，我们碰到的问题2，这是ref2.ApproximateSize做实际上近似于缓存的大小了可怕的工作。通过CLR垃圾艰难地，我发现，这是一个System.SizedReference，这是它在做什么来获取值（IntPtr的是一个句柄的MemoryCache对象本身）：
[SecurityCritical]
[MethodImpl（MethodImplOptions.InternalCall）
私人静态外部长GetApproximateSizeOfSizedRef（IntPtr的H）;
我假设外部声明意味着它去潜水非托管窗口的土地在这一点上，我不知道如何下手找出它做什么那里。从我所观察到的，虽然它试图逼近的整体大小事一个可怕的工作。
第三个引人注目的还有这听起来像它应该做的事的呼叫manager.UpdateCacheSize。遗憾的是在本应是如何工作的s_memoryCacheManager任何正常的样本将始终为NULL。该字段从公共静态构件ObjectCache.Host设置。这是通过喷溅起来我自己IMemoryCacheManager实施，将它设置为ObjectCache.Host，然后运行样品暴露用户乱用，如果他选择这样做，而我居然能做出这个东西梳理工作就像它应该。在这一点上，虽然，它好像你可能也仅仅使自己的缓存实现，甚至没有用这些东西麻烦，尤其是因为我不知道，如果设定自己的类来ObjectCache.Host（静态，所以它影响每个人这些可能是在那里过程）来衡量缓存可以搞乱其他东西。
我不得不相信，至少一部分的这种（如果不是一对夫妇的部分）仅仅是一个直线上升的bug。它会是不错的一个人在MS什么问题是这个东西听。
TLDR版本，这个巨大的答案：假设CacheMemoryLimitMegabytes是在这个时间点上完全捣毁。你可以将其设置为10 MB，然后进行填充缓存〜2GB和打击的内存溢出异常，无跳闸项删除。
I’m working with the .NET 4.0 MemoryCache class in an application and trying to limit the maximum cache size, but in my tests it does not appear that the cache is actually obeying the limits.
I'm using the settings which, according to MSDN, are supposed to limit the cache size:
- CacheMemoryLimitMegabytes: The maximum memory size, in megabytes, that an instance of an object can grow to."
- PhysicalMemoryLimitPercentage: "The percentage of physical memory that the cache can use, expressed as an integer value from 1 to 100. The default is zero, which indicates that MemoryCache instances manage their own memory1 based on the amount of memory that is installed on the computer." 1. This is not entirely correct-- any value below 4 is ignored and replaced with 4.
I understand that these values are approximate and not hard limits as the thread that purges the cache is fired every x seconds and is also dependent on the polling interval and other undocumented variables. However even taking into account these variances, I'm seeing wildly inconsistent cache sizes when the first item is being evicted from the cache after setting CacheMemoryLimitMegabytes and PhysicalMemoryLimitPercentage together or singularly in a test app. To be sure I ran each test 10 times and calculated the average figure.
These are the results of testing the example code below on a 32-bit Windows 7 PC with 3GB of RAM. Size of the cache is taken after the first call to CacheItemRemoved() on each test. (I am aware the actual size of cache will be larger than this)
MemLimitMB MemLimitPct AVG Cache MB on first expiry
1 NA 84
2 NA 84
3 NA 84
6 NA 84
NA 1 84
NA 4 84
NA 10 84
10 20 81
10 30 81
10 39 82
10 40 79
10 49 146
10 50 152
10 60 212
10 70 332
10 80 429
10 100 535
100 39 81
500 39 79
900 39 83
1900 39 84
900 41 81
900 46 84
900 49 1.8 GB approx. in task manager no mem errros
200 49 156
100 49 153
2000 60 214
5 60 78
6 60 76
7 100 82
10 100 541
Here is the test application:
using System;
using System.Collections.Generic;
using System.Collections.Specialized;
using System.Linq;
using System.Runtime.Caching;
using System.Text;
namespace FinalCacheTest
{
internal class Cache
{
private Object Statlock = new object();
private int ItemCount;
private long size;
private MemoryCache MemCache;
private CacheItemPolicy CIPOL = new CacheItemPolicy();
public Cache(long CacheSize)
{
CIPOL.RemovedCallback = new CacheEntryRemovedCallback(CacheItemRemoved);
NameValueCollection CacheSettings = new NameValueCollection(3);
CacheSettings.Add("CacheMemoryLimitMegabytes", Convert.ToString(CacheSize));
CacheSettings.Add("physicalMemoryLimitPercentage", Convert.ToString(49)); //set % here
CacheSettings.Add("pollingInterval", Convert.ToString("00:00:10"));
MemCache = new MemoryCache("TestCache", CacheSettings);
}
public void AddItem(string Name, string Value)
{
CacheItem CI = new CacheItem(Name, Value);
MemCache.Add(CI, CIPOL);
lock (Statlock)
{
ItemCount++;
size = size + (Name.Length + Value.Length * 2);
}
}
public void CacheItemRemoved(CacheEntryRemovedArguments Args)
{
Console.WriteLine("Cache contains {0} items. Size is {1} bytes", ItemCount, size);
lock (Statlock)
{
ItemCount--;
size = size - 108;
}
Console.ReadKey();
}
}
}
namespace FinalCacheTest
{
internal class Program
{
private static void Main(string[] args)
{
int MaxAdds = 5000000;
Cache MyCache = new Cache(1); // set CacheMemoryLimitMegabytes
for (int i = 0; i < MaxAdds; i++)
{
MyCache.AddItem(Guid.NewGuid().ToString(), Guid.NewGuid().ToString());
}
Console.WriteLine("Finished Adding Items to Cache");
}
}
}
Why is MemoryCache not obeying the configured memory limits?
Wow, so I just spent entirely too much time digging around in the CLR with reflector, but I think I finally have a good handle on what's going on here.
The settings are being read in correctly, but there seems to be a deep-seated problem in the CLR itself that looks like it will render the memory limit setting essentially useless.
The following code is reflected out of the System.Runtime.Caching DLL, for the CacheMemoryMonitor class (there is a similar class that monitors physical memory and deals with the other setting, but this is the more important one):
protected override int GetCurrentPressure()
{
int num = GC.CollectionCount(2);
SRef ref2 = this._sizedRef;
if ((num != this._gen2Count) && (ref2 != null))
{
this._gen2Count = num;
this._idx ^= 1;
this._cacheSizeSampleTimes[this._idx] = DateTime.UtcNow;
this._cacheSizeSamples[this._idx] = ref2.ApproximateSize;
IMemoryCacheManager manager = s_memoryCacheManager;
if (manager != null)
{
manager.UpdateCacheSize(this._cacheSizeSamples[this._idx], this._memoryCache);
}
}
if (this._memoryLimit <= 0L)
{
return 0;
}
long num2 = this._cacheSizeSamples[this._idx];
if (num2 > this._memoryLimit)
{
num2 = this._memoryLimit;
}
return (int) ((num2 * 100L) / this._memoryLimit);
}
The first thing you might notice is that it doesn't even try to look at the size of the cache until after a Gen2 garbage collection, instead just falling back on the existing stored size value in cacheSizeSamples. So you won't ever be able to hit the target right on, but if the rest worked we would at least get a size measurement before we got in real trouble.
So assuming a Gen2 GC has occurred, we run into problem 2, which is that ref2.ApproximateSize does a horrible job of actually approximating the size of the cache. Slogging through CLR junk I found that this is a System.SizedReference, and this is what it's doing to get the value (IntPtr is a handle to the MemoryCache object itself):
[SecurityCritical]
[MethodImpl(MethodImplOptions.InternalCall)]
private static extern long GetApproximateSizeOfSizedRef(IntPtr h);
I'm assuming that extern declaration means that it goes diving into unmanaged windows land at this point, and I have no idea how to start finding out what it does there. From what I've observed though it does a horrible job of trying to approximate the size of the overall thing.
The third noticeable thing there is the call to manager.UpdateCacheSize which sounds like it should do something. Unfortunately in any normal sample of how this should work s_memoryCacheManager will always be null. The field is set from the public static member ObjectCache.Host. This is exposed for the user to mess with if he so chooses, and I was actually able to make this thing sort of work like it's supposed to by slopping together my own IMemoryCacheManager implementation, setting it to ObjectCache.Host, and then running the sample. At that point though, it seems like you might as well just make your own cache implementation and not even bother with all this stuff, especially since I have no idea if setting your own class to ObjectCache.Host (static, so it affects every one of these that might be out there in process) to measure the cache could mess up other things.
I have to believe that at least part of this (if not a couple parts) is just a straight up bug. It'd be nice to hear from someone at MS what the deal was with this thing.
TLDR version of this giant answer: Assume that CacheMemoryLimitMegabytes is completely busted at this point in time. You can set it to 10 MB, and then proceed to fill up the cache to ~2GB and blow an out of memory exception with no tripping of item removal.
这篇关于的MemoryCache不服从配置的内存限制的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
