缓存时NGINX内存峰值 [英] NGINX memory spiking when caching

问题描述

我对NGINX并不陌生，并尝试设置缓存，尽管我能够进行缓存工作，但我遇到了内存使用问题.现在，在NGINX文档中，它说NGINX可以在1MB空间中存储大约8000个缓存密钥.因此，通过配置，我运行了具有20K个唯一请求的负载测试.很快，我发现NGINX盒上的内存使用量猛增到70MB.现在，根据文档，我可以想象到内存使用量比我所看到的要低得多.另外，我确认仅在打开缓存时内存使用量才会增加.

I am fairly new to NGINX and trying to setup caching, although I am able to get caching work I am running into a memory usage problem. Now in the NGINX documentation it says that NGINX can store around 8000 cache keys within 1MB space. So with the config I have I ran a load test with 20K unique requests. Very quickly I see that the memory usage on the NGINX boxes spiked to 70MB. Now as per the documentation I would imagine that the memory usage to be much lower than what I am seeing. Also I confirmed that the memory usage is spiking only when I turn on caching.

这是我的NGINX配置的缓存部分:

Here is the caching part of my NGINX config:

proxy_cache_path /tmp/cache keys_zone=static:4000m max_size=10g inactive=12h;
proxy_cache static;
add_header X-Cache-Status $upstream_cache_status;
proxy_ignore_headers X-Accel-Expires Expires Set-Cookie;
proxy_ignore_headers Cache-Control;
proxy_cache_min_uses 1;
proxy_cache_key '$scheme$host$uri';
proxy_cache_valid 200 12h;

有人可以帮忙吗?

谢谢.

推荐答案

这似乎是Windows上Nginx的问题.

This seems to be a problem of Nginx on Windows.

在Linux上没有发生这种现象，按预期，它每1Mb keys_zone大小可缓存约8K个请求.

On Linux this behavior is not experienced, it caches ~8K requests per 1Mb keys_zone size, as expected.

在Windows上，缓存的行为有所不同.

On Windows cache behaves differently.

注意 Nginx网站上的评论:

需要共享内存支持的缓存和其他模块 在Windows Vista和更高版本上无法使用.."

"The cache and other modules which require shared memory support do not work on Windows Vista and later versions .."

如果不调查源代码，即使有足够的空间，我什至不能说带有内存共享的缓存在Windows上是否可以正常工作.

Without looking into sources, I can't even say whether the cache with memory sharing is working properly on Windows at all, even given enough space.

根据我的测试，直到达到以下文件计数限制为止，它会按预期从缓存中投放. Windows区域中的每个键分配可能需要4K(页面大小).

From my tests, until you hit file count limits below, it serves from cache as expected. It probably requires 4K (page size) for each key allocation in zone on Windows.

我测试了25K个请求，缓存中保留的最大文件数量与keys_zone的大小成正比，如下所示:

I tested with 25K requests and the maximum number of files that are retained in cache is proportional to the keys_zone size as follows:

1m  <= 253 files 
2m  <= 508 files 
3m  <= 761 
5m  <= 1274 
10m <= 2548 
20m <= 5103 
40m <= ~8K 
80m <= ~20K

(数字代表1.9.3、1.11.6表现出相似的行为，但数字略有不同).

(numbers are for 1.9.3, 1.11.6 exposes similar behavior with slightly different numbers).

这与您的内存增长观察结果一致.一旦达到上述每个keys_zone大小的文件数限制，它就会很奇怪地丢失.

This is consistent with your memory growth observation. Once you reach file count limit per keys_zone size as above, it is going to miss weirdly.

在Ubuntu上，Nginx 1.4和1.10确实会根据限制创建预期的缓存文件数；正如预期的那样，每8k请求的25K高速缓存请求的进程内存增长不会超过〜1Mb.

On Ubuntu, Nginx 1.4 and 1.10 do create expected number of cache files with respect to limit; process memory for 25K cached requests does not grow far beyond ~1Mb per 8k requests, as expected.

这篇关于缓存时NGINX内存峰值的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！