WINdows 7 上的高频数据侦听器 TCP 过载 [英] High Frequency Data Listener TCP Overload on WIndows 7
问题描述
我编写了一个基于 c# 的侦听器来从股票市场服务器检索数据.问题是，通过 WireShark 进行监控后，TCP 窗口已满，这意味着接收主机(我)无法足够快地处理传入数据.
我知道我的数据提供者已经禁用了他这边的 TCP 确认，所以基本上他只是推送 TCP 数据包并查看它们是否仍然存在":如果他检测到它们在一定时间后没有被处理，连接就会被简单地关闭.
我真的不知道该怎么办，我已经禁用了 Windows 7 的自动缩放调整和启发式，但这没有任何效果.我还注意到无法在 Windows 7 上设置 TPC Windows 大小 (RWIN).
我知道这不是一个与编程相关的问题本身但它在某种程度上是，因为从代码的角度来看这很关键.
编辑
响应者表示:您的应用程序提取数据的速度不够快"这很有趣，但我不知道如何优化它:
私有类 IoContext{//用于操作的套接字:公共套接字_ipcSocket;//用于操作的缓冲区:公共 VfxMsgBlock _ipcBuffer;公共 IoContext(套接字套接字，VfxMsgBlock 缓冲区){_ipcSocket = 套接字；_ipcBuffer = 缓冲区；}}私有无效InitiateRecv(IoContext rxContext){rxContext._ipcSocket.BeginReceive(rxContext._ipcBuffer.Buffer, rxContext._ipcBuffer.WrIndex,rxContext._ipcBuffer.Remaining(), 0, CompleteRecv, rxContext);}私有无效 CompleteRecv(IAsyncResult ar){IoContext rxContext = ar.AsyncState as IoContext;如果(rxContext ！= null){int rxBytes = rxContext._ipcSocket.EndReceive(ar);如果(rxBytes > 0){//调整消息块中的写索引:rxContext._ipcBuffer.WrIndex = rxContext._ipcBuffer.WrIndex + rxBytes;//(...) 在这里用数据做 StufrxContext._ipcBuffer.Crunch();//启动另一个异步读取:InitiateRecv(rxContext);}}}编辑 2
回应 Len Holgate:
我直接通过 c# 属性更改/检查了接收缓冲区大小(使用 GetSocketOption/SetSocketOption 时遇到了一些麻烦)
当我改变 Socket Buffer 大小时，我可以注意到轻微的性能变化，即连接在 1000 万后下降，使用大缓冲区 (1000000) 而不是 300 万(使用 10240 或更少(甚至 0，我不知道为什么... 奇怪))
这是我得到的第一个 WireShark 日志:
[SYN, ACK] Seq=0 Ack=1 Win=5840 Len=0 MSS=1380 SACK_PERM=1 WS=128它不会改变 Socket 的接收缓冲区大小
关于 Windows 7 &TCP Windows 大小，我的意思是在 Windows 中重写 TCP 管理导致它自动缩放，并且没有可用的注册表参数来手动"设置它.
但是，您是对的:在每个套接字的基础上，它可以使用接收缓冲区大小进行调整.
来源此处等
您是否已将套接字的接收缓冲区设置得尽可能大?
请参阅此处了解如何影响 TCP 窗口大小:http://msdn.microsoft.com/en-us/library/ms819736.aspx
编辑以显示我通过 1gb 链接使用两台 win 7 机器得到的结果，没有更改默认设置，只需使用 setsockopt()<设置 SO_RECVBUF/code> 在侦听套接字上，从连接的wireshark 日志中验证了窗口大小...实际窗口大小是根据广告大小和缩放因子隐含的乘数计算得出的.
设置接收 |广告 |缩放|乘法|实际 TCP |差异|泡到 |窗口 ||按 |窗口大小 ||--------+-----------+-------+--------+----------- +----------+默认 |8192 |8 |第256话2097152 ||1024 |1024 |0 |1 |1024 |0|10240 |10240 |0 |1 |10240 |0|66666666 |65535 |10 |1024 |67107840 |-441174|666666666 |65535 |14 |16384 |1073725440 |-407058774|1000000000 |65535 |14 |16384 |1073725440 |-73725440|1073725440 |65535 |14 |16384 |1073725440 |0|1073741823 |65535 |14 |16384 |1073725440 |16383|1073741824 |8192 |8 |第256话2097152 |1071644672|2147483647 |8192 |8 |第256话2097152 |2145386495|所以，很明显，在 Windows 7 上有可能在每个套接字的基础上影响 TCP 窗口大小.你能否提供一个指向文档的链接，让你相信从 Windows 7 开始，我刚刚说过不再有可用于手动设置的 TCP 窗口大小(仅限自动调整 netsh 选项)".
请注意，在发出连接之前，必须将 recv buf 设置应用于侦听套接字或出站套接字，因为窗口大小缩放在打开握手中传输.
在所有情况下，对 setsockopt() 的调用都成功了，并且对 getsockopt() 的立即调用报告了 recv 缓冲区的大小是前一次调用中请求的大小setsockopt().
请注意，尽管理论上的最大值是 unsigned int 的最大值 2147483647，但实际的最大有用值似乎是 1073741823 (0x3FFFFFFF)，这可能因操作系统版本而异，但请记住，立即调用 getsockopt() 返回与设置相同的值，因此不可能确定 1073741823 以上的值被完全忽略(至少从窗口大小的角度来看).
当然，这需要连接点指定一个窗口缩放选项，甚至是 0，以表明它接受窗口缩放.
请向我们展示初始握手数据包的wireshark连接日志.我可以提供编译后的代码，你可以测试这是你想要的.
I have coded a c# based listener to retrieve data from a stock market server.
The problem is, after monitoring through WireShark, TCP window gets full, meaning that the receiving host (me) has trouble processing incoming data fast enough.
I Know my data provider has disabled TCP acknowledgement from his side, so basically he just pushes TCP Packets and looks if they are "still there" : If he detects that they were not processed after a certain amount of time, connection is simply shut down.
I really don't know what to do, Ive disabled Windows 7's Auto Scale Tuning and also Heuristics, but that did not have any effect. I also noticed that there is no way to set TPC Windows Size (RWIN) on Windows 7.
I Know this is not a programming related question per se but it is somehow, because this is critical from the code point of view.
EDIT
A responder stated : "your application is not pulling data out of it fast enough"
This is fairly interesting, but I have no real idea on how to optimize this :
private class IoContext
{
// The socket used for the operation:
public Socket _ipcSocket;
// The buffer used for the operation:
public VfxMsgBlock _ipcBuffer;
public IoContext(Socket socket, VfxMsgBlock buffer)
{
_ipcSocket = socket;
_ipcBuffer = buffer;
}
}
private void InitiateRecv(IoContext rxContext)
{
rxContext._ipcSocket.BeginReceive(rxContext._ipcBuffer.Buffer, rxContext._ipcBuffer.WrIndex,
rxContext._ipcBuffer.Remaining(), 0, CompleteRecv, rxContext);
}
private void CompleteRecv(IAsyncResult ar)
{
IoContext rxContext = ar.AsyncState as IoContext;
if (rxContext != null)
{
int rxBytes = rxContext._ipcSocket.EndReceive(ar);
if (rxBytes > 0)
{
//Adjust the write index in the message block:
rxContext._ipcBuffer.WrIndex = rxContext._ipcBuffer.WrIndex + rxBytes;
//(...) Do Stuf here with data
rxContext._ipcBuffer.Crunch();
//Initiate another asynchronous read:
InitiateRecv(rxContext);
}
}
}
EDIT 2
In response to Len Holgate :
I changed/checked the Recv Buffer size directly via c# property (got some troubles using GetSocketOption/SetSocketOption)
I can notice a slight performance change when I vary the Socket Buffer size, ie connection drops after 10mn with a big bufer (1000000) rather than 3mn (using 10240 or less (even 0, I don't know why... weird))
Here is the first WireShark log I am getting :
[SYN, ACK] Seq=0 Ack=1 Win=5840 Len=0 MSS=1380 SACK_PERM=1 WS=128
It doesn't change whatever the Socket's receive buffer size
About Windows 7 & TCP Windows size, I meant than the re-write of TCP Management in Windows resulted in it being scaled automatically, and no registry parameter is available to set it "by hand".
However, you are right : On a per-socket basis, it is adjustable using the receive buffer size.
Source here among others
Have you set the socket's recv buffer to as large as you can?
See here for how to affect the TCP window size: http://msdn.microsoft.com/en-us/library/ms819736.aspx
Edited to show what I get with two win 7 machines over a 1gb link, no changes to default settings, just setting SO_RECVBUF using setsockopt() on the listen socket, window size verified from the wireshark log of the connection... The actual window size is calculated from the advertised size and the multiplier that is implied by the scaling factor.
Set Recv |Advertised |Scaling|Multiply| Actual TCP |Difference| Buf to | Window | | By |Window size | | --------+-----------+-------+--------+----------- +----------+ default | 8192 | 8 | 256 | 2097152 | | 1024 | 1024 | 0 | 1 | 1024 | 0| 10240 | 10240 | 0 | 1 | 10240 | 0| 66666666 | 65535 | 10 | 1024 | 67107840 | -441174| 666666666 | 65535 | 14 | 16384 | 1073725440 |-407058774| 1000000000 | 65535 | 14 | 16384 | 1073725440 | -73725440| 1073725440 | 65535 | 14 | 16384 | 1073725440 | 0| 1073741823 | 65535 | 14 | 16384 | 1073725440 | 16383| 1073741824 | 8192 | 8 | 256 | 2097152 |1071644672| 2147483647 | 8192 | 8 | 256 | 2097152 |2145386495|
So, clearly, it IS possible to affect the TCP Window size on a per socket basis on Windows 7. Could you provide a link to the document that makes you believe that "As of windows 7 I just said that there is no TCP Window Size available for manual setting anymore (auto-tune netsh option only)" please.
Note that the recv buf setting must be applied to the listening socket, or the outbound socket before you issue the connect as the window size scaling is transmitted in the opening handshake.
In all cases the call to setsockopt() succeeded and an immediate call to getsockopt() reported the recv buffer to be the size that was requested in the previous call to setsockopt().
Note that although the theoretical maximum is the max value of an unsigned int, 2147483647, the actual maximum useful value appears to be 1073741823 (0x3FFFFFFF), this may vary with operating system version but bear in mind that an immediate call to getsockopt() returned the same value that had been set so it was impossible, except by looking at the wireshark log, to determine that values above 1073741823 were ignored entirely (at least from a window size perspective).
Of course this requires that the connecting peer is specifying a window scale option, even 0, to show that it accepts window scaling.
Please show us the wireshark connection log for the initial handshake packets. I can provide compiled code that you can you to test this is you want.
这篇关于WINdows 7 上的高频数据侦听器 TCP 过载的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
