可以持久化到磁盘的 memcached 的替代方案 [英] alternative to memcached that can persist to disk

问题描述

我目前正在将 memcached 与我的 Java 应用程序一起使用，总体而言它运行良好.

对我来说最重要的 memcached 功能是:

I am currently using memcached with my java app, and overall it's working great.

The features of memcached that are most important to me are:

• 速度很快，因为读取和写入都在内存中并且不接触磁盘
• 它只是一个键/值存储(因为这是我的应用程序的全部需求)
• 它是分布式的
• 它通过让每个对象仅存在于一台服务器上来有效地使用内存
• 它不假设对象来自数据库(因为我的对象不是数据库对象)

• it's fast, since reads and writes are in-memory and don't touch the disk
• it's just a key/value store (since that's all my app needs)
• it's distributed
• it uses memory efficiently by having each object live on exactly one server
• it doesn't assume that the objects are from a database (since my objects are not database objects)

但是，有一件我想做而 memcached 做不到的事情.我想定期(也许每天一次)将缓存内容保存到磁盘.我希望能够从保存的磁盘映像中恢复缓存.

磁盘保存不需要非常复杂.如果在保存时添加了新的键/值，我不在乎它是否包含在保存中.如果在保存时修改了现有的键/值，保存的值应该是旧值或新值，但我不在乎是哪一个.

任何人都可以推荐另一种缓存解决方案(免费或商业)，它具有对我很重要的所有(或很大比例)的 memcached 功能，并且还允许从磁盘保存和恢复整个缓存的能力?

However, there is one thing that I'd like to do that memcached can't do. I want to periodically (perhaps once per day) save the cache contents to disk. And I want to be able to restore the cache from the saved disk image.

The disk save does not need to be very complex. If a new key/value is added while the save is taking place, I don't care if it's included in the save or not. And if an existing key/value is modified while the save is taking place, the saved value should be either the old value or the new value, but I don't care which one.

Can anyone recommend another caching solution (either free or commercial) that has all (or a significant percentage) of the memcached features that are important to me, and also allows the ability to save and restore the entire cache from disk?

推荐答案

也许你的问题和我的一样:我只有几台机器用于 memcached，但内存很大.即使其中之一出现故障或需要重新启动，也会严重影响系统的性能.根据最初的 memcached 理念，我应该为每个机器添加更多内存更少的机器，但这不符合成本效益，也不完全是绿色 IT"；)

Maybe your problem is like mine: I have only a few machines for memcached, but with lots of memory. Even if one of them fails or needs to be rebooted, it seriously affects the performance of the system. According to the original memcached philosophy I should add a lot more machines with less memory for each, but that's not cost-efficient and not exactly "green IT" ;)

对于我们的解决方案，我们为缓存系统构建了一个接口层，以便底层缓存系统的提供者可以嵌套，就像您可以对流一样，并为 memcached 编写了一个缓存提供者以及我们自己非常简单的 Key-Value-2-disk 存储提供程序.然后我们为缓存项定义一个权重，该权重表示如果无法从缓存中检索项，则重建该项的成本.嵌套的磁盘缓存仅用于权重高于特定阈值的项目，可能占所有项目的 10% 左右.

For our solution, we built an interface layer for the Cache system so that the providers to the underlying cache systems can be nested, like you can do with streams, and wrote a cache provider for memcached as well as our own very simple Key-Value-2-disk storage provider. Then we define a weight for cache items that represent how costly it is to rebuild an item if it cannot be retrieved from cache. The nested Disk cache is only used for items with a weight above a certain threshold, maybe around 10% of all items.

将对象存储在缓存中时，我们不会浪费时间，因为保存到一个或两个缓存中的任何内容都会排队等待异步执行.所以写入磁盘缓存不需要很快.读取也是一样:首先我们选择 memcached，并且只有当它不存在并且它是一个昂贵"的对象时，然后我们检查磁盘缓存(它比 memcached 慢很多，但仍然比重新计算 30 GB 好得多单机宕机后的数据).

When storing an object in the cache, we won't lose time as saving to one or both caches is queued for asynchronous execution anyway. So writing to the disk cache doesn't need to be fast. Same for reads: First we go for memcached, and only if it's not there and it is a "costly" object, then we check the disk cache (which is by magnitudes slower than memcached, but still so much better then recalculating 30 GB of data after a single machine went down).

这样我们就可以两全其美，而无需用任何新东西替换 memcached.

This way we get the best from both worlds, without replacing memcached by anything new.

这篇关于可以持久化到磁盘的 memcached 的替代方案的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！