TCP 接收窗口大小高于 net.core.rmem_max [英] TCP receiving window size higher than net.core.rmem_max
问题描述
我正在通过 10Gbit 链路连接的两台服务器之间运行 iperf 测量.我正在尝试将我观察到的最大窗口大小与系统配置参数相关联.
I am running iperf measurements between two servers, connected through 10Gbit link. I am trying to correlate the maximum window size that I observe with the system configuration parameters.
特别是，我观察到最大窗口大小为 3 MiB.但是，我在系统文件中找不到相应的值.
In particular, I have observed that the maximum window size is 3 MiB. However, I cannot find the corresponding values in the system files.
通过运行 sysctl -a 我得到以下值:
By running sysctl -a I get the following values:
net.ipv4.tcp_rmem = 4096 87380 6291456
net.core.rmem_max = 212992
第一个值告诉我们最大接收器窗口大小为 6 MiB.然而，TCP 倾向于分配两倍于请求的大小，因此最大接收器窗口大小应该是 3 MiB，正如我所测量的那样.来自 man tcp:
The first value tells us that the maximum receiver window size is 6 MiB. However, TCP tends to allocate twice the requested size, so the maximum receiver window size should be 3 MiB, exactly as I have measured it. From man tcp:
请注意，TCP 实际分配的缓冲区大小是 setsockopt(2) 调用中请求的缓冲区大小的两倍，因此后续的 getockopt(2) 调用将不会返回与 setsockopt(2) 调用中请求的缓冲区大小相同的缓冲区.TCP 将额外空间用于管理目的和内部内核结构，与实际 TCP 窗口相比，/proc 文件值反映了更大的大小.
Note that TCP actually allocates twice the size of the buffer requested in the setsockopt(2) call, and so a succeeding getsockopt(2) call will not return the same size of buffer as requested in the setsockopt(2) call. TCP uses the extra space for administrative purposes and internal kernel structures, and the /proc file values reflect the larger sizes compared to the actual TCP windows.
然而，第二个值 net.core.rmem_max 表明最大接收器窗口大小不能超过 208 KiB.根据 man tcp 的说法，这应该是硬限制:
However, the second value, net.core.rmem_max, states that the maximum receiver window size cannot be more than 208 KiB. And this is supposed to be the hard limit, according to man tcp:
tcp_rmemmax:每个 TCP 套接字使用的接收缓冲区的最大大小.此值不会覆盖全局net.core.rmem_max.这不用于限制在套接字上使用 SO_RCVBUF 声明的接收缓冲区的大小.
tcp_rmem max: the maximum size of the receive buffer used by each TCP socket. This value does not override the global
net.core.rmem_max. This is not used to limit the size of the receive buffer declared using SO_RCVBUF on a socket.
那么，为什么我观察到的最大窗口大小大于 net.core.rmem_max 中指定的大小?
So, how come and I observe a maximum window size larger than the one specified in net.core.rmem_max?
注意:我还计算了带宽延迟乘积:window_size = Bandwidth x RTT，大约为 3 MiB(10 Gbps @ 2 毫秒 RTT)，从而验证了我的流量捕获.
NB: I have also calculated the Bandwidth-Latency product: window_size = Bandwidth x RTT which is about 3 MiB (10 Gbps @ 2 msec RTT), thus verifying my traffic capture.
推荐答案
出现了快速搜索:
在void tcp_select_initial_window()
if (wscale_ok) {
/* Set window scaling on max possible window
* See RFC1323 for an explanation of the limit to 14
*/
space = max_t(u32, sysctl_tcp_rmem[2], sysctl_rmem_max);
space = min_t(u32, space, *window_clamp);
while (space > 65535 && (*rcv_wscale) < 14) {
space >>= 1;
(*rcv_wscale)++;
}
}
max_t 取参数的较高值.所以这里较大的值优先.
max_t takes the higher value of the arguments. So the bigger value takes precedence here.
另外一个对 sysctl_rmem_max 的引用被用来将参数限制为 SO_RCVBUF(在 net/core/sock.c 中).
One other reference to sysctl_rmem_max is made where it is used to limit the argument to SO_RCVBUF (in net/core/sock.c).
所有其他 tcp 代码仅指 sysctl_tcp_rmem.
All other tcp code refers to sysctl_tcp_rmem only.
因此，无需深入研究代码，您就可以得出结论，在所有情况下，更大的 net.ipv4.tcp_rmem 将覆盖 net.core.rmem_max>SO_RCVBUF(可以使用SO_RCVBUFFORCE绕过其检查)
So without looking deeper into the code you can conclude that a bigger net.ipv4.tcp_rmem will override net.core.rmem_max in all cases except when setting SO_RCVBUF (whose check can be bypassed using SO_RCVBUFFORCE)
这篇关于TCP 接收窗口大小高于 net.core.rmem_max的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
