如何使用Redis缓存来缓存大对象 [英] How to cache large objects using Redis cache

问题描述

我们当前的缓存实现将大量数据缓存在报表对象中（某些情况下为50MB）。

我们已经从内存缓存转移到文件缓存，并使用ProtoBuf进行序列化和反序列化。这很好用，但是我们现在正在尝试Redis缓存。
以下是Redis与使用文件系统相比需要花费多长时间的示例。 （注意：在下面的示例中，设置字节数组时，使用protobuf代替JsonConvert可以将设置时间缩短为15秒，将时间缩短为4秒。）

  //极慢-使用Redis进行缓存（通过JsonConvert进行序列化/反序列化）
 IDatabase缓存= Connection.GetDatabase（）; 
 
 // 23秒！ 
 cache.StringSet（ myKey，JsonConvert.SerializeObject（bigObject））; 
 
 // 5秒！ 
 BigObject redisResult = JsonConvert.DeserializeObject< BigObject>（cache.StringGet（ myKey）））; 
 
 
 
 
 //快速-使用文件系统进行缓存（protobuf进行序列化/反序列化）
 IDataAccessCache fileCache = new DataAccessFileCache（）; 
 
 // .5秒
 fileCache.SetCache（ myKey，bigObject）; 
 
 // .5秒
 BigObject fileResult = fileCache.GetCache< BigObject>（ myKey）; 
  

在此先感谢您的帮助。

ps。我没有从类似的问题中找到答案。
缓存大型对象-LocalCache性能

或

缓存大对象，减少检索时间的影响

解决方案

Redis实际上不是设计好的用于存储大对象（许多MB），因为它是单线程服务器。因此，一个请求将足够快，但是一些请求将变得很慢，因为它们全部将由一个线程处理。在最后一个版本中，我们做了一些优化。对于大对象（> 10 KB），它可能会变得很明显。通常，购买昂贵的快速内存模块来优化Redis并不真正具有成本效益。 https://redis.io/topics/benchmarks

因此，您可以使用超大帧或购买更快的内存。但是实际上，它并没有太大帮助。
考虑改用 Memcached 。它是多线程的，可以水平扩展以支持大量数据。只能通过主从复制来缩放Redis。

Our current caching implementation caches large amounts of data in report objects (50MB in some cases).

We’ve moved from memory cache to file cache and use ProtoBuf to serialize and de-serialize. This works well, however we are now experimenting with Redis cache. Below is an example of how much longer it takes for Redis than using the file system. (Note: using protobuf instead of JsonConvert improves set time to 15 seconds and get time to 4 seconds in the below example, when setting a byte array).

// Extremely SLOW – caching using Redis (JsonConvert to serialize/de-serialize)
IDatabase cache = Connection.GetDatabase();

// 23 seconds!
cache.StringSet("myKey", JsonConvert.SerializeObject(bigObject));

// 5 seconds!
BigObject redisResult = JsonConvert.DeserializeObject<BigObject>(cache.StringGet("myKey")); 




// FAST - caching using file system (protobuf to serialize/de-serialize)
IDataAccessCache fileCache = new DataAccessFileCache();

// .5 seconds
fileCache.SetCache("myKey",bigObject); 

// .5 seconds                                          
BigObject fileResult = fileCache.GetCache<BigObject>("myKey");                              

Thanks in advance for any help.

ps. I didn’t find an answer from similar questions asked. Caching large objects - LocalCache performance

or

Caching large objects, reducing impact of retrieval times

解决方案

Redis actually is not designed for storing large objects (many MB) because it is a single-thread server. So, one request will be fast enough, but a few requests will be slow because they all will be processed by one thread. In the last versions some optimizations were done.

Speed of RAM and memory bandwidth seem less critical for global performance especially for small objects. For large objects (>10 KB), it may become noticeable though. Usually, it is not really cost-effective to buy expensive fast memory modules to optimize Redis. https://redis.io/topics/benchmarks

So, you can use Jumbo frames or buy a faster memory if it is possible. But actually it won't help significantly. Consider using Memcached instead. It is multi-threaded and can be scaled out horizontally to support large amount of data. Redis can be scaled only with master-slave replication.

这篇关于如何使用Redis缓存来缓存大对象的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！