Nginx代理缓冲-更改缓冲区的数量与大小? [英] Nginx proxy buffering - changing buffer's number vs size ?
问题描述
我想知道并试图弄清楚这两个设置如何:
I was wondering and trying to figure out how these two settings:
proxy_buffers [number] [size];
可能会影响(改善/降低)代理服务器的性能，以及是否更改缓冲区的大小，数量或两者兼而有之??
may affect (improve / degrade) proxy server performance, and whether to change buffers' size, or the number, or both...?
在我的特定情况下，我们所谈论的是一个系统，它服务于动态生成的二进制文件，该文件的大小可能会有所不同(〜60-200kB). Nginx在两个充当生成器的Tomcat之前充当负载平衡器.我在Nginx的error.log中看到，使用默认缓冲区的大小设置，所有代理响应都缓存到文件中，因此，我发现合乎逻辑的是将设置更改为以下内容:
In my particular case, we're talking about a system serving dynamically generated binary files, that may vary in size (~60 - 200kB). Nginx serves as a load-balancer in front of 2 Tomcats that act as generators. I saw in Nginx's error.log that with default buffers' size setting all of proxied responses are cached to a file, so what I found to be logical is to change the setting to something like this:
proxy_buffers 4 32k;
，警告消息消失.
这里我不清楚，我是否应该设置1个较大的缓冲区或几个较小的缓冲区...例如:
What's not clear to me here is if I should preferably set 1 buffer with the larger size, or several smaller buffers... E.g.:
proxy_buffers 1 128k;与proxy_buffers 4 32k;与proxy_buffers 8 16k;等
可能有什么区别，以及它如何影响性能(如果有的话)?
What could be the difference, and how it may affect performance (if at all)?
推荐答案
首先，最好查看文档中有关指令的内容:
First, it's a good idea to see what the documentation says about the directives:
http://nginx.org/r/proxy_buffers
语法: proxy_buffers 数字大小；
默认值:proxy_buffers 8 4k | 8k;
上下文:http，服务器，位置
Syntax: proxy_buffers number size;
Default: proxy_buffers 8 4k|8k;
Context: http, server, location
设置单个连接用于从代理服务器读取响应的缓冲区的数量和大小.默认情况下，缓冲区大小等于一个内存页.根据平台的不同，它可以是4K或8K.
Sets the number and size of the buffers used for reading a response from the proxied server, for a single connection. By default, the buffer size is equal to one memory page. This is either 4K or 8K, depending on a platform.
将级别升级到 http://nginx.org/r/proxy_bufferring 可以提供更多的功能说明:
Jumping a level up to http://nginx.org/r/proxy_bufferring provides a bit more explanation:
启用缓冲后，nginx会尽快从代理服务器收到响应，并将其保存到proxy_buffer_size和proxy_buffers指令设置的缓冲区中.如果整个响应都无法容纳到内存中，则可以将一部分响应保存到磁盘上的临时文件中. …
When buffering is enabled, nginx receives a response from the proxied server as soon as possible, saving it into the buffers set by the proxy_buffer_size and proxy_buffers directives. If the whole response does not fit into memory, a part of it can be saved to a temporary file on the disk. …
当禁用缓冲时，响应在接收到后立即立即同步传递到客户端. …
When buffering is disabled, the response is passed to a client synchronously, immediately as it is received. …
那么，这一切意味着什么?
So, what does all of that mean?
-
首先，对Nginx进行了高度优化，使其最有效地利用了资源.众所周知，使用最少的资源来服务每个单独的连接.每个连接额外增加4KB将会是一个相当大的增长-这就是效率.
First of all, Nginx is highly optimised to be the most efficient with the resources at stake. It's well known for using the least amount of resources to service each individual connection. An extra 4KB increase per connection would be quite an increase — that's how efficient it is.
您可能会注意到，缓冲区的大小被选择为等于所关注平台的页面大小. https://en.wikipedia.org/wiki/Page_(computer_memory)基本上，长话短说，绝对最好的数字可能也超出了StackOverflow上该问题的范围，并且可能高度依赖于操作系统和CPU架构.
You may notice that the size of the buffer is chosen to be equivalent to page size of the platform at stake. https://en.wikipedia.org/wiki/Page_(computer_memory) Basically, long story short, the absolute best number might as well go beyond the scope of this question on StackOverflow, and may as well be highly dependent on operating system and CPU architecture.
实际上，较大数量的较小缓冲区或较小数量的较大缓冲区之间的差异可能取决于操作系统提供的内存分配器，以及您拥有多少内存以及多少内存您想通过分配而浪费内存，而没有将其用于良好目的.
Realistically, the difference between a bigger number of smaller buffers, or a smaller number of bigger buffers, may depend on the memory allocator provided by the operating system, as well as how much memory you have, and how much memory you want to be wasted by being allocated without being used for a good purpose.
例如，我不会将其设置为proxy_buffers 1 1024k，因为那样您将为每个缓冲的连接分配1MB的缓冲区，即使内容仍然可以轻松地容纳在4KB的缓冲区中，也可能浪费额外的内存(当然，还有一个鲜为人知的事实自1980年代以来，未使用但已分配的内存实际上是免费的).将缓冲区的默认数量也选择为8可能是一个很好的理由.
E.g., I would not set it to proxy_buffers 1 1024k, because then you'll be allocating a 1MB buffer for every buffered connection, even if the content could still easily fit in a 4KB one, potentially wasting extra memory (although, of course, there's also the little-known fact that unused-but-allocated-memory is virtually free since 1980s). There's likely a good reason that the default number of buffers have been chosen to be 8 as well.
Increasing the buffers at all might actually be a bit pointless if you actually do caching of the responses of these binary files with http://nginx.org/r/proxy_cache, because nginx will still be writing it to disc for caching, and you might as well not waste the extra memory for buffering these responses.
一个好的操作系统应该已经能够对通过文件系统缓冲区高速缓存功能写入磁盘的内容进行适当的高速缓存-请参阅 https://en.wikipedia.org/wiki/Page_cache 上的文章，称为磁盘缓冲区" "HDD硬件文章中已经使用了该名称")-因此，几乎不需要在nginx中直接复制缓冲.您也可以查看 varnish-cache ，以获取有关多层次主题的一些其他想法和启发.缓存，事实是良好的操作系统应该已经很好地照顾了许多事情，但是有些人仍然尝试通过特定于应用程序的功能错误地进行优化.
A good operating system should be capable of already doing appropriate caching of the stuff that gets written to disc through the file-system buffer-cache functionality — see https://www.tldp.org/LDP/sag/html/buffer-cache.html (and possibly the somewhat strangely-named article at https://en.wikipedia.org/wiki/Page_cache, as "disk-buffer" name was already taken for the HDD hardware article) — so, there's likely little need to duplicate buffering directly within nginx. You might also take a look at varnish-cache for some additional ideas and inspiration about the subject of multi-level caching, and the fact that good operating systems are already supposed to take good care of many things that some folks still nonetheless try to mistakenly optimise through application-specific functionality.
同样，如果您实际上不缓存响应，那么您最好先问一下自己是否适合缓存.
Likewise, if you don't actually do the caching of the responses, then you might as well ask yourself whether or not buffering is or is not appropriate in the first place.
实际上，缓冲可能会有用，可以更好地保护您的上游免受慢速洛里斯攻击媒介的侵害- https: //en.wikipedia.org/wiki/Slowloris_(computer_security)-但是，如果您确实让nginx具有兆字节大小的缓冲区，那么从本质上讲，您将开始暴露nginx本身，因为它消耗了不合理的数量资源以恶意目的为客户提供服务.
Realistically, buffering may come useful to better protect your upstreams from the slowloris attack vector — https://en.wikipedia.org/wiki/Slowloris_(computer_security) — however, if you do let your nginx have megabyte-sized buffers, then, essentially, you're starting to expose nginx itself for consuming an unreasonable amount of resources to service clients with malicious intents.
如果响应太大，则可能需要考虑在响应级别对其进行优化.将某些内容拆分为单个文件；在文件级别进行压缩；在HTTP Content-Encoding级别上使用gzip进行压缩等.
If the responses are too large, you might want to look into optimising it at the response level. Doing splitting of some content into individual files; doing compression on the file level; doing compression with gzip on HTTP Content-Encoding level etc.
TL; DR:这确实是一个相当广泛的问题，并且有太多的变量需要进行简单的调查才能针对任何给定的情况得出绝对最佳的答案.
TL;DR: this is really a pretty broad question, and there are too many variables that require non-trivial investigation to come up with the absolute best answer for any given situation.
这篇关于Nginx代理缓冲-更改缓冲区的数量与大小?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
