将所有内存缓存密钥转储到文件中的最简单方法是什么? [英] What's the simplest way to get a dump of all memcached keys into a file?
问题描述
这仅来自具有大约2000万个密钥(没有到期)和大约2G数据的单个Memcached服务器.
This is from just a single memcached server with around 20M keys (no expiry) and around 2G of data.
将所有键/值对转储到平面文件中的最简单方法是什么?我首先看了java net.spy.memcached.MemcachedClient，但是此客户端不支持获取所有密钥(我认为).如果我有所有键的列表(我没有)，则可以轻松地使用此客户端获取所有值.
What's the easiest way to get a dump of all the key/value pairs into a flat file? I first looked at the java net.spy.memcached.MemcachedClient, but this client does not support getting all keys (I think). If I had a list of all keys (which I don't), I could easily use this client to get all of the values.
我知道我可以使用某些telnet命令(例如telnet localhost 11211；统计信息项；统计信息cachedump)来获取所有密钥，但是我不清楚如何可靠地自动执行此操作.
I know I can get all keys using some telnet commands (e.g., telnet localhost 11211; stats items; stats cachedump ), but it's not clear to me how to automate this robustly.
这是我在我的机器上的玩具内存缓存服务器上进行此操作的工作.看来可行，但我只在memcached中放了两个键，因此希望该方法可以扩展:
Here's what I did to get this working on a toy memcached server on my machine. It seems to work but I only put two keys in memcached, so hopefully this method will scale ok:
shell命令:
sudo yum install memcached
sudo /etc/init.d/memcached restart # maybe unnecessary
sudo yum install php
sudo yum install php-pecl-memcache
sudo service httpd reload
php脚本，基于此:
php script, based on this:
<?php
$memcache = new Memcache();
$memcache->connect('127.0.0.1', 11211) or die ("Could not connect");
$list = array();
$allSlabs = $memcache->getExtendedStats('slabs');
$items = $memcache->getExtendedStats('items');
foreach($allSlabs as $server => $slabs) {
foreach($slabs AS $slabId => $slabMeta) {
if (!is_int($slabId)) {
continue;
}
$cdump = $memcache->getExtendedStats('cachedump', (int) $slabId, 100000000);
foreach($cdump AS $server => $entries) {
if ($entries) {
foreach($entries AS $eName => $eData) {
print_r($eName);
print_r(":");
$val = $memcache->get($eName);
print_r($val);
print_r("\n");
}
}
}
}
}
?>
上面的脚本似乎没有返回所有映射.如果我插入行count($entries)，即使将limit参数设置为100M，它也仅返回50k多一点，但是从telnet执行stats items则显示超过5M条目.有谁知道为什么会这样?
The above script does not seem to return all of the mappings. If I insert the line count($entries), it only returns a little over 50k, even with the limit parameter set to 100M, but executing stats items from telnet shows over 5M entries. Does anyone know why this might be the case?
此> 链接表明，cachedump无法从memcached获取所有密钥.我已经达到了大约5万个键的限制，这些键是由cachedump，此PHP脚本或类似于Zach Bonham提供的链接中的一个perl脚本返回的.有什么办法解决吗?
This link suggests that cachedump doesn't get all keys from memcached. I've hit a limit of around 50k keys that are returned either by cachedump, this PHP script, or a perl script similar to one in the link provided by Zach Bonham. Is there any way around this?
推荐答案
免责声明:我不知道自己在做什么，听起来像是一个有趣的问题.
您看到这篇文章了吗? 如何从Memcache中转储密钥" ，作者是Lars Windolf.
Did you see this article? "How to Dump Keys from Memcache" by Lars Windolf.
摘自文章:
Memcache本身提供了进入数据峰值的方法.协议 提供命令以探查由平板组织的数据 (给定大小范围的数据类别.有些重要 局限性:
Memcache itself provide the means to peak into the data. The protocol provides commands to peak into the data that is organized by slabs (categories of data of a given size range. There are some significant limitations though:
- 您只能转储每个slab类的密钥(具有大致相同内容大小的密钥)
- 每个平板类只能转储一页(1MB数据)
- 这是一个非官方功能，随时可以删除.
- You can only dump keys per slab class (keys with roughly the same content size)
- You can only dump one page per slab class (1MB of data)
- This is an unofficial feature that might be removed anytime.
有效地，它需要一些有关内存缓存如何将数据存储在内存中的知识(我不知道).您需要找到每个"slab"，然后可以转储该slab的密钥，然后最终转储那些密钥的值.
Effectively, it requires some knowledge of how memcache stores data in memory (which I don't). You need to find each 'slab', then you can dump the keys for that slab, and then ultimately, the values for those keys.
本文中有一个工具部分，该部分使用各种语言来转储至少键，但是只有perl脚本会转储键和值.
There is a tools section in the article which uses various languages to dump at least the keys, but only the perl script dumps both keys and values.
这篇关于将所有内存缓存密钥转储到文件中的最简单方法是什么?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
