Redis内存和CPU峰值 [英] redis memory and cpu spikes

问题描述

我们对应用程序中的某些数据使用redis，这非常好。但是，我注意到 redis-server 进程中偶尔出现CPU和内存峰值。

We use redis for some data in our app, and it's totally great. I noticed however occasional cpu and memory spikes on the redis-server process.

这是长颈鹿仪表板。分期显然不那么忙，但是生产通常也不是很忙...

This is the Giraffe dashboard from both our production and staging environments. Staging is obviously much less busy, but production isn't terribly busy either normally...

这似乎与节省背景相关，但并非与所有节省相关。其中只有少数会造成这种峰值。也许所有方法都可以，但这仅取决于测量分辨率（有些根本不在我们的内存/ CPU监视周期之内）。我不确定。

This seems to correlate with the background saving, but not with all of them. Only a handful of those create this spike. Perhaps all do, and it's only down to the measurement resolution though (some are simply not caught within our memory/cpu monitoring cycle). I'm not entirely sure.

我仍然想知道这是否正常/正常。我们没有发现任何问题，但我想保持安全。如果我们在生产中有更多的流量/活动，是否可能还会看到更多类似的峰值？

I'm still wondering whether this is expected / normal. We don't observe any issues, but I want to be on the safe side. If we have more traffic/activity on our production, are we likely to see many more spikes like these?

更新：

高峰时间前后的redis日志文件

redis log file around the time of the spike

[18588] 05 May 11:42:51.004 * 10 changes in 300 seconds. Saving...
[18588] 05 May 11:42:51.258 * Background saving started by pid 32712
[32712] 05 May 11:43:00.511 * DB saved on disk
[32712] 05 May 11:43:00.549 * RDB: 1 MB of memory used by copy-on-write
[18588] 05 May 11:43:00.629 * Background saving terminated with success

推荐答案

通过对此进行进一步的试验并阅读有关 redis持久性，我认为可以得出以下结论：

From experimenting further with this and reading about redis persistence, I think the following observations can be made:

• 使用RDB（默认设置）时，每次触发保存操作（默认情况下，设置为每15分钟最低。当执行更多对Redis的写入操作时，RDB写入频率每 60秒一次。


• 每个fork将使用写时复制内存分配，这意味着虽然内存实际上不会增加​​一倍-它会在 ps ， htop 之类的工具中显示


• fork本身可能是CPU密集型操作，尤其是在基于xen的虚拟主机上（这是我们当前正在使用的虚拟主机）。


• 写操作似乎完全覆盖了现有RDB 文件。它不仅写入更改，而且将 entire 数据集转储到磁盘。

• When using RDB (the default settings), redis will fork each time a save operation is triggered, which (by default) is set to once every 15 minutes as a minimum. When more writes to Redis are performed, then RDB writes are as frequent as once every 60 seconds.
• Each fork will use a "copy-on-write" memory allocation, which means that whilst memory won't actually double - it will appear so on tools like ps, htop and the like.
• The fork itself can be quite a cpu-intensive operation, particularly on xen-based virtual hosts (which is what we're using currently).
• The write operation seems to completely overwrite the existing RDB file. It does not write only the changes, but rather dumps the entire dataset to disk.

适中的虚拟主机，具有4Gb RAM和大约750Mb的数据集（在我发布问题时），这开始变得相当昂贵。即使在相当适度的负载/ redis使用情况下，我们也观察到了CPU /内存峰值以及IO的增加。

So on a modest virtual host with 4Gb RAM and data set of around 750Mb (at the time I posted the question), this starts to become rather "expensive". We observed those CPU/Memory spikes, as well as increased IO, even under fairly moderate load / redis usage.

所以回答我自己的问题-这似乎是

So to answer my own question - this does seem to be the "expected" behaviour.

为了改善这种情况，我们选择切换配置以使用RDB和AOF的组合。 AOF（仅附加文件）似乎确实只将更改写入磁盘。您仍然可以（并且应该）将AOF文件配置为重写（使用 auto-aof-rewrite-percentage 和 auto-aof-rewrite-min-大小设置）。建议仍将RDB用于快照。但是，在此配置中，您可能可以不那么频繁地执行完全重写/快照，并且仍然可以保持相当不错的性能以及更好的耐用性。

As for improving the situation, we opted to switch our configuration to use a combination of RDB and AOF. AOF (Append Only File), does appear to only write changes to disk. You can (and should) still configure the AOF file to rewrite (using auto-aof-rewrite-percentage and auto-aof-rewrite-min-size settings). It is also advisable to still use RDB for snapshots. In this configuration however, you can probably do full rewrites / snapshots less frequently and still maintain pretty-good performance and even-better durability.

这篇关于Redis内存和CPU峰值的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！