使用IOCP的TCP / IP服务器。接收缓冲区中偶尔的数据损坏 [英] TCP/IP server using IOCP. Occasional data corruption in receive buffers
问题描述
我一直在使用TCP / IP IOCP服务器应用程序。
我一直在测试性能（看起来符合TCP吞吐量测试实用程序）现在已经测试数据完整性 - 这是我得到一些怪异。
作为一个初步测试，我决定让一个测试客户端发送一个1MB的块数据，其中该块仅仅是一个接一个递增的整数序列。这个想法是，我可以验证每个接收的数据缓冲区是否与该缓冲区中没有丢失的数据一致，独立于接收到的任何其他缓冲区，这意味着我不需要担心什么顺序线程处理完成的接收。 （要验证，我提取缓冲区中的第一个整数并向前扫描，如果我遇到客户端发送的最大值，将期望值重置为0我还有一个检查，以确保每个接收的数据是4的倍数（因为它们是4个字节的整数））。
我似乎得到一个缓冲区中偶尔丢失的随机数据块，值将上升1乘1，那么就会有一堆被跳过。代码虽然看起来很简单，但不是很多。我最初在Delphi中编写测试，但是在遇到这些问题后，我在Visual Studio 2010 C ++中重写了一个版本，似乎有相同的问题（或至少非常相似）。
在真正的系统中显然有更多的代码，但是我可以在工作线程中将其归类，只处理完成的接收，验证缓冲区中的数据，然后再次发布。在我最初接受连接之后，我创建两个重叠结构，并为每个缓冲区分配1MB缓冲区，然后为每个缓冲区调用WSARecv。我已经双重检查我不是不小心在两者之间共享相同的缓冲区。然后，下面几乎是运行重用这些的：
DWORD numberOfBytesTransferred = 0;
ULONG_PTR completionKey = NULL;
PMyOverlapped overlapped = nullptr;
while（true）
{
auto queueResult = GetQueuedCompletionStatus（iocp，& numberOfBytesTransferred，& completionKey，（LPOVERLAPPED *）& overlapped，INFINITE）;
if（queueResult）
{
switch（overlapped-> operation）
{
case tsoRecv：
{
verifyReceivedData numberOfBytesTransferred）; //检查数据是一个递增的整数1的序列，没有gabs
overlapped-> overlapped = OVERLAPPED（）; //将OVERLAPPED结构重置为缺省值
DWORD flags = 0;
numberOfBytesTransferred = 0;
auto returnCode = WSARecv（socket，&（overlapped-> buffer），1，& numberOfBytesTransferred，& flags，（LPWSAOVERLAPPED）overlapped，nullptr）;
break;
}
default :;
}
}
}
种类的错误或额外的信息在我简单的测试上面？我最初有一个IOCP客户端发送数据，但在Delphi中使用Indy阻塞套接字写了另一个非常简单的数据。它基本上是一行代码一旦连接。
while true do
begin
IdTCPClient.IOHandler.WriteDirect （TIdBytes（BigData），Length（BigData））;
end;
我还使用不同的异步套接字组件编写了另一个服务器，我没有检测到问题接收数据作为我的IOCP示例上面，至少还。我可以发布更多的代码和可能的一个版本编译，但认为我会发布上面的情况下，我错过了一些明显的东西。我想使用一个接收和一个发送每个套接字工作确定，但我的理解是，有效的发布多个以提高性能。
我相信这是解决了 - 我的大部分假设和代码是正确的，但似乎对于一个特定的套接字，不能同时从多个线程调用WSASend或WSARead。对于特定的套接字，可以有多个未完成的呼叫用于发送和接收，但是启动它们的实际呼叫需要用关键部分（或类似部分）串行化。这是一个对我的MSDN文档的一点误解，我认为可以做，但你不知道哪个缓冲区将首先填充没有一些额外的同步（和我的测试不在乎首先填充）。它似乎它根本不安全，除非调用一次一个，并可能导致缓冲区内的数据损坏。
我改变的唯一的代码是每个连接添加一个关键部分来保护这些呼叫，到目前为止没有任何问题。我认为可能单独保护WSASend和WSARecv，但还没有测试。
我已经发布了一个更深入的问题与这个这里有更多的代码示例。 p>
I’ve been working on an TCP/IP IOCP server application.
I’ve been testing performance (which seems in line with TCP throughput testing utilities) and now have been testing data integrity – this is where I am getting some "weirdness".
As an initial test, I decided to have a test client send a 1MB block of data over and over where that block is just a sequence of integers incremented one after the other. The idea being that I can verify that each received buffer of data is consistent with no missing data in that buffer, independent from any other buffer received, meaning I don't need to worry what order threads handle the completed receives. (to verify, I extract the first integer in the buffer and scan forward, reset the expected value to 0 if I encounter the max value the client sends. I also have a check to make sure the data received in each is a multiple of 4 (as they are 4 byte integers)).
I seem to be getting random blocks of data missing occasionally from within a buffer, the values will go up 1 by 1, then there will be a bunch skipped. The code though seems quite simple and not much of it. I originally wrote the test in Delphi but after having these issues I rewrote a version in Visual Studio 2010 C++ and seem to be having the same issue (or at least very similar).
There is obviously more code in the real system, but I can boil it down to pretty much this in the worker thread, that just handles the completed receives, verifies the data in the buffer and then posts them again. After I initially accept the connection, I create two overlapped structures and allocate 1MB buffers to each and then call WSARecv for each of these. I've double checked I'm not accidentally sharing the same buffer between the two. Then, the following is pretty much what runs reusing these:
DWORD numberOfBytesTransferred = 0;
ULONG_PTR completionKey = NULL;
PMyOverlapped overlapped = nullptr;
while (true)
{
auto queueResult = GetQueuedCompletionStatus(iocp, &numberOfBytesTransferred, &completionKey, (LPOVERLAPPED *)&overlapped, INFINITE);
if (queueResult)
{
switch (overlapped->operation)
{
case tsoRecv:
{
verifyReceivedData(overlapped, numberOfBytesTransferred); // Checks the data is a sequence of incremented integers 1 after the other with no gabs
overlapped->overlapped = OVERLAPPED(); // Reset the OVERLAPPED structure to defaults
DWORD flags = 0;
numberOfBytesTransferred = 0;
auto returnCode = WSARecv(socket, &(overlapped->buffer), 1, &numberOfBytesTransferred, &flags, (LPWSAOVERLAPPED) overlapped, nullptr);
break;
}
default:;
}
}
}
Maybe I am not handling some kind of error or additional information in my simple test above? I originally had an IOCP client sending data but wrote another extremely simple one in Delphi using Indy blocking sockets. It's basically a line of code once connected.
while true do
begin
IdTCPClient.IOHandler.WriteDirect(TIdBytes(BigData), Length(BigData));
end;
I also wrote another server using a different asynchronous socket component and I haven't had it detect problems with the received data as my IOCP example above does, at least yet. I can post more code and possibly a version to compile but thought I would post the above in case I've missed something obvious. I think using one receive and one send per socket works ok, but it's my understanding that it's valid to post more than one to improve performance.
I believe this is solved - most of my assumptions and code were correct however it seems that for a particular socket, there cannot be simultaneous calls from multiple threads to WSASend or WSARead. There can be multiple outstanding calls for both sends and receives for a particular socket but the actual calls to initiate them need to serialized with a critical section (or similar). This was a slight misunderstanding of the MSDN documentation on my part, I was thinking it could be done but you wouldn't know which buffer would be filled first without some additional synchronization (and my test didn't care which got filled first). It appears it's simply not safe at all unless the calls are made one at a time and can result in corrupted data within the buffers.
The only code I have changed is to add a critical section per connection to protect calls to these and so far have had no issues. I think it might be possible to protect WSASend and WSARecv separately but have not tested that yet.
I had posted a far more in-depth question related to this here that has more code examples.
这篇关于使用IOCP的TCP / IP服务器。接收缓冲区中偶尔的数据损坏的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
