Redis缓慢获取大字符串 [英] Redis is slow to get large strings

问题描述

我对Redis有点陌生，所以如果这是一个愚蠢的问题，我深表歉意。

I'm kind of a newb with Redis, so I apologize if this is a stupid question.

我正在将Django与Redis用作缓存。

I'm using Django with Redis as a cache.

我正在腌制约200个对象的集合并将其存储在Redis中。

I'm pickling a collection of ~200 objects and storing it in Redis.

当我请求该集合时来自Redis的Django Debug Toolbar通知我，对Redis的请求大约需要3秒钟的时间。我必须做一些可怕的错误。

When I request the collection from Redis, Django Debug Toolbar is informing me that the request to Redis is taking ~3 seconds. I must be doing something horribly wrong.

• 服务器具有3.5GB的内存，看起来Redis当前仅在使用〜50mb，所以我很确定它不会用完内存。

• The server has 3.5GB of ram, and it looks like Redis is currently using only ~50mb, so I'm pretty sure it's not running out of memory.

当我得到时使用redis-cli的密钥所花的时间与我从Django

When I get the key using the redis-cli it takes just as long as when I do it from Django

运行 strlen 关于redis-cli的密钥，我得知长度约为2000万（这太大了吗？）

Running strlen on the key from redis-cli I'm informed that the length is ~20 million (Is this too large?)

如何使Redis更快地返回数据？如果这看起来不寻常，可能会有一些常见的陷阱？我已经看到此页面有关延迟问题，但是还没有真正跳出来的东西。

What can I do to have Redis return the data faster? If this seems unusual, what might be some common pitfalls? I've seen this page on latency problems, but nothing has really jumped out at me yet.

我不确定将大量数据存储在一个密钥中是否真的是个坏主意，或者我的配置是否有问题。

I'm not sure if it's a really bad idea to store a large amount of data in one key, or if there's just something wrong with my configuration. Any help or suggestions or things to read would be greatly appreciated.

推荐答案

Redis并非用于存储非常大的对象。您不应该将整个集合存储在Redis中的单个字符串中，而是使用Redis列表或将其设置为对象的容器。

Redis is not designed to store very large objects. You are not supposed to store your entire collection in a single string in Redis, but rather use Redis list or set as a container for your objects.

此外，泡菜格式未针对空间进行优化...您将需要更紧凑的格式。协议缓冲区，MessagePack甚至纯JSON可能对此更好。您应该考虑在存储数据（例如Snappy，LZO，Quicklz，LZF等）之前应用光压缩算法。

Furthermore, the pickle format is not optimized for space ... you would need a more compact format. Protocol Buffers, MessagePack, or even plain JSON, are probably better for this. You should consider to apply a light compression algorithm before storing your data (like Snappy, LZO, Quicklz, LZF, etc ...).

最后，性能可能是网络绑定。在我的机器上，从Redis检索20 MB对象需要85毫秒（而不是3秒）。现在，如果我使用远程服务器运行相同的测试，则需要1.781秒，这在此100 Mbit / s网络上是可以预期的。持续时间完全取决于网络带宽。

Finally, the performance is probably network bound. On my machine, retrieving a 20 MB object from Redis takes 85 ms (not 3 seconds). Now, if I run the same test using a remote server, it takes 1.781 seconds, which is expected on this 100 Mbit/s network. The duration is fully dependent on the network bandwidth.

最后一点：请确保使用最新的Redis版本-已经进行了许多优化来处理大型对象。

Last point: be sure to use a recent Redis version - a number of optimization have been done to deal with large objects.

这篇关于Redis缓慢获取大字符串的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！