密钥到期之前有Memcached逐出吗? [英] Memcached eviction prior to key expiry?

问题描述

如果仍有可用空间，存储在memcached中的键/值对是否可以在到期之前被驱逐?

Can a key/value pair stored in memcached get evicted prior to its expiry if there is still free space available?

我有一个正在运行的memcached进程，该进程最多可以消耗6GB；据报告使用了2.5GB，并且该数字仅发生很小的波动(一日跨度为+/- 100MB).如果我设置了一个在15分钟后失效的简单字符串值，是否有可能在经过15分钟后将其逐出(找不到cache.get返回值)?

I have a memcached process running that is allowed to consume up to 6GB; 2.5GB are reported in use and that number fluctuates only minimally (+/- 100MB in a one-day span). If I set a simple string value that expiries in 15 minutes, is it possible that it would be evicted (cache.get returns not found) prior to 15 minutes elapsing?

谢谢， -埃里克(Eric)

Thanks, -Eric

推荐答案

是

基本上，memcache会按块与按需分配内存，然后将项目存储到块中并手动管理该内存.结果，与逐项分配空间相比，较小的项可以使用"更大的内存.

Basically, memcache allocates space in chuncks vs on-demand, and then stores items into the chunks and manages that memory manually. As a result, smaller items can "use" much larger pieces of memory than they would if space was allocated on a per-item basis.

该链接比我能更好地解释它

The link explains it much better than I can

https://groups.google.com/group/memcached/browse_thread/thread /8f460034418262e7?pli = 1

修改:添加更多说明

Memcache通过分配各种大小的平板来工作.这些平板具有许多特定大小的插槽(由平板的类别确定).

Memcache works by allocating slabs of various sizes. These slabs have a number of specifically sized slots (which is determined by the slab's class).

假设(并且仅使用我对Memcache内部的抽象)，可以说最小尺寸的平板类为1K.这意味着最小的插槽为1K.此外，Memcache只会一次分配1024个或1MB的内存.可以说我们有这样的配置，我们想将1个字节的对象(char值?)存储到Memcache中.让我们假设这将需要5个字节的内存(4个字节的密钥?).在空的缓存中，Memcache会分配一个可以容纳该值(1K插槽)的最小大小的新平板.因此，存储5个字节将导致Memcache分配1MB的内存.

Hypothetically (and using only my abstraction of Memcache's internals), lets say the smallest size slab class was 1K. This means that the smallest slots are 1K. Furthermore, Memcache will only allocate these in sets of 1024, or 1MB of memory at a time. Lets say we had such a configuration and we want to store a 1-byte object (char value?) into Memcache. Lets suppose this would require 5 bytes of memory (4 byte key?). In an empty cache, Memcache would allocate a new slab of the smallest size that can hold the value (1K slots). So storing your 5 bytes will cause Memcache to allocate 1MB of memory.

现在，假设您有很多这样的东西.下一个1023将是免费"的-内存缓存已经分配了内存，因此不需要其他内存.最后，您已经存储了1024 * 5字节=〜5KB，但是Memcache使用了1MB来存储它.存储其中的几百万，您可以想象要消耗数千兆字节的内存来存储千字节的数据.

Now, let say you have a lot of these. The next 1023 will be "free" -- Memcache has already allocated the memory, so no additional memory is needed. At the end of this, you've stored 1024 * 5 bytes = ~5KB, but Memcache has used 1MB to store this. Store a few million of these, and you can imagine consuming gigabytes of memory to store kilobytes of data.

这几乎是最坏的情况.在实践中，如果需要，可以将Memcache配置为具有最小的最小Slab类大小，并且可以扩大或缩小增长因子(Slab类之间的大小差异).如果要缓存数据库查询，则项目的大小可能从几字节到几KB，甚至还有页面内容，甚至可以进入MB.

This is close to a worst case. In practice Memcache can be configured to have a minimum slab class size quite small if needed, and the growth factor (size difference between the slab-classes) can be widened or narrowed. If you're caching database queries, you might have items sized from a few bytes to several KB, with page content you could even get into the MB.

这是要点 Memcache不会回收内存或清理平板(新版本现在确实具有此功能，对性能的影响很大，但传统上，这是Memcache的工作方式).

Here's the key point Memcache won't reclaim memory or clean up slabs (new versions do have this now for a pretty significant performance hit, but traditionally, this has been how Memcache works).

假设您的系统已经愉快地运行并缓存了几天.您有数百个各种大小的平板.您可以在不重置缓存的情况下将新的页面缓存策略部署到您的应用程序.现在，您要缓存部分页面，而不是缓存整个页面.您已将缓存模式从存储约1MB的对象更改为存储约10KB的对象.这就是我们遇到麻烦的地方. Memcache已分配了一堆可容纳约1MB对象的平板.您以前从未缓存过许多10KB的对象.具有10KB插槽的slab很快就会被填满，但是现在您有一堆已分配的slab，它们容纳了1MB未被使用的对象(其他都没有那么大). Memcache不会将10KB的对象放在1MB的插槽中(即使这样做，也不会很长时间).它需要获得更多容纳10KB对象的平板，但是不能，因为您的所有内存都已分配到容纳1MB对象的平板中.结果是，您可能在平板中分配了可能的千兆字节内存，以容纳1MB对象，这些对象在10KB插槽的平板已满时处于空闲状态.在这种情况下，尽管闲置了千兆字节，您仍将开始从10KB插槽的平板中逐出项目.

Suppose you have a system that has been happily running and caching for a few days. You have hundreds of slabs of various sizes. You deploy a new page-caching strategy to your app without resetting the cache. Now instead of caching whole pages, you're caching parts of the page. You've changed your caching pattern from storing lots of ~1MB objects to storing lots of ~10KB objects. Here's where we get into trouble. Memcache has allocated a bunch of slabs that hold objects of about 1MB. You never used to cache many 10KB objects before. The slabs that have 10KB slots are quickly filled up, but now you have a whole bunch of allocated slabs that hold objects of 1MB which aren't being used (nothing else is that big). Memcache won't put your 10KB objects in a 1MB slot (even if it did, it wouldn't help for very long). It needs to get more slabs that hold 10KB objects, but it can't because all your memory has been allocated into the slabs that hold 1MB objects. The result is that you are left with potentially gigabytes of memory allocated in slabs to hold 1MB objects which sit idle while your 10KB-slot slabs are full. In this scenario, you will start evicting items out of the 10KB-slot slabs despite have gigabytes sitting idle.

这是一个漫不经心，人为的极端例子.您的缓存策略很少会如此明显或如此显着地变化. slab-class的默认增长因子是1.25，因此您将拥有1KB插槽，1.25KB插槽，1.5KB插槽等的slab.该概念适用-如果您大量使用某些大小的slab，并且模式会发生变化(sql查询返回更多对象?网页变大?在表中添加一列，以将缓存的响应移至slab类上?等等.)然后您可以得到一堆大小不正确的slab，并且可以尽管有千兆字节的未使用"空间，但无处"存储东西.

This was a long-winded, contrived, and extreme example. Rarely does your caching strategy change so obviously or so dramatically. The default growth factor of slab-classes is 1.25, so you'd have slabs with 1KB slots, 1.25KB slots, 1.5KB slots, etc. The concept holds -- if you are heavily using certain sized slabs and that pattern shifts (sql queries return more objects? web pages get bigger? add a column to a table which moves a cached response up a slab class? etc.) Then you can end up with a bunch of slabs which are the "wrong" size and you can have "nowhere" to store something despite having gigabytes of "unused" space.

如果您要搬迁，可以通过telnet进入Memcache并找出导致搬迁的平板.通常，缓存重置(是的，清空所有内容)可以解决此问题.这是有关如何获取统计信息的参考. http://lzone.de/articles/memcached.htm

If you are getting evictions, it's possible to telnet into Memcache and find out what slabs are causing the evictions. Usually, a cache-reset (yeah, empty out everything) fixes the issue. Here's a reference on how to get at the stats. http://lzone.de/articles/memcached.htm

这篇关于密钥到期之前有Memcached逐出吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！