I/O完成端口* LAST *称为回调，或:在安全的地方清理事物 [英] I/O Completion Ports *LAST* called callback, or: where it's safe to cleanup things

问题描述

我想这个论点很重要，应该在这里留一些空间.

I guess this argument is important and deserve some space here.

让我们考虑C/C ++中最常见的I/O完成端口设计， 具有抽象处理HANDLE及其某些属性的结构(或类)，如下所示:

Let's consider the most common design of I/O Completion Ports in C/C++, having a structure (or a class) which abstract the HANDLE, and some of its properties, like this:

class Stream 
{
    enum
    {
         Open = 1,
         Closed = 0
    };

    // Dtor
    virtual ~Stream() { if (m_read_packet != 0) delete_packet(m_read_packet); // the same for write packet }


    // Functions:
    bool read(...) { if (m_read_packet != 0) m_read_packet = allocate_packet(); ReadFile(m_handle ...); }
    bool write(...);
    bool close() { m_state = Closed; if (!CloseHandle(m_handle)) return false; else return true; }


    // Callbacks:
    virtual void onRead(...);
    virtual void onWrite(...);
    virtual void onEof();
    virtual void onClose();


    // Other stuff ....


    // Properties:
    HANDLE m_handle;
    int m_state;
    int m_pending_reads;
    int m_pending_writes;
    IoPacket * m_read_packet;
    IoPacket * m_write_packet;
};

这非常简单，您有一个抽象HANDLE的类，并且 I/O完成时，IO完成端口线程将调用其回调.另外，您可以缓存 并重用存储在这些指针中的OVERLAPPED结构(由IoPacket派生).

This is pretty straightforward, you have this class which abstract an HANDLE, and the IO Completion Port threads call its callbacks when i/o is completed. Plus, you can cache and reuse the OVERLAPPED structures (derived by IoPacket) stored in those pointers.

您可以通过自己分配Stream对象或仅让其他类分配来使用此类 连接到达后(例如AcceptEx()或命名管道服务器).流对象将 根据其回调实现自动开始执行I/O操作.

You use this class by allocating a Stream object yourself, or just by let other classes to allocate it, after a connection arrives (e.g. AcceptEx(), or a named pipe server). The Stream object will automatically start to do i/o operation based on its callbacks implementation.

现在，在两种情况下，您都选择[1)自己分配流(客户端)或2)从其他对象(服务器)分配流] 这将是一个常见的问题:您完全不知道何时可以安全地从内存中取消分配Stream对象.即使对IoPacket对象使用原子引用计数器，也是如此.

Now, in both cases you choose [1) allocate Stream yourself (client) or 2) from other objects (server)] there will be a common problem: you don't know exactly when you can safely deallocate from memory the Stream object. And this is true even if you use atomic ref counters for IoPacket objects.

让我们看看为什么:做I/o时，Stream对象指针将包含在IoPacket对象中，该对象依次由线程池中的其他线程处理和使用，线程池中的其他线程又使用该指针来调用回调[ onRead()，onWrite()等...].您还可以使用"m_pending_reads/writes"变量来查看有多少未决读/写.因此，Stream对象指针将是跨线程共享的.

Let's see why: while doing i/o, the Stream object pointer would be contained in the IoPacket objects, which in turn are processed and used by other threads in the threadpool, which in turn uses that pointer to call the callbacks [onRead(), onWrite() and so on...]. You have also `m_pending_reads/writes' variables to see how many pending reads/writes there are. So, the Stream object pointer will be shared accross threads.

在这一点上，让我们考虑您不再需要该连接，并且您想要关闭并取消分配相关的Stream对象.

At this point, let's consider you don't need that connection anymore and you want to close and deallocate the related Stream object.

如何安全地执行此操作?

我的意思是，如果您只是这样做:

I mean, if you simply do:

stream->close();
delete stream;

您将最终陷入崩溃，因为另一个线程可能仍在使用'stream'指针. 安全的方法是这样:

you'll end up in some crashes because another thread may still be using the 'stream' pointer. The safe way would be this:

stream->close();
while ((stream->m_pending_reads != 0) && (stream->m_pending_writes != 0));
delete stream;

但这是完全无效的，因为这会阻塞执行路径g，阻塞等待其他线程 用流"对象完成工作. (此外，我想我还会添加一些易失性或屏障机制?) 最好的方法是异步执行此操作，另一个线程为该Stream对象完成 LAST IoPacket，然后调用onClose()，依次为:

But this is utterly inefficent since this blocks the path of executiong, block-waiting that other threads finish their work with the 'stream' object. (And in addition I think I'll have add some volatile or barrier mechanism?) The best way would be do that asynchronously, with another thread completing the LAST IoPacket for that Stream object, and call onClose() which in turn will be:

void onClose() { delete this; }

所以我的问题是:如何识别我正在处理(在任何线程中)给定句柄的 LAST IoPacket(或OVERLAPPED)，所以在我可以安全地调用onClose()之后删除Stream对象的例程，然后dtor依次删除用于执行I/O任务的缓存的IoPackets.

So my question is: how to recognize that I'm processing (in any thread) the LAST IoPacket (or OVERLAPPED) for the given handle, so after I could call safely the onClose() routine which deletes the Stream objects, which dtor in turn deletes the cached IoPackets, used for doing i/o stuff.

现在我使用这种方法:如果(对于阅读补全)"GetQueuedCompletionStatus()"返回ERROR_OPERATION_ABORTED [因此，它表示CloseHandle(m_handle); [已经被调用] OR bytes_read == 0 [这表示EOF]或(state！= Open)则表示它是最后一个挂起的读取数据包，然后调用onClose();经过处理.写入数据包将忽略onClose().

For now I use this method: if (for reading completions) `GetQueuedCompletionStatus()' returns ERROR_OPERATION_ABORTED [so it means CloseHandle(m_handle); has been called] OR bytes_read==0 [it means EOF] OR if (state!=Open) THEN it means that it was the last reading packet pending, then calling onClose(); after processing that. The write packets will ignore onClose().

我不确定这是正确的方法，因为即使读取的数据包是 始终处于待处理状态，直到发生I/O事件为止，否则可能发生在计划在另一个愿意处理最后一个写数据包的线程之前的某个线程调用onClose()的情况.

I'm not sure that this is the correct method, because even if the reading packets are those which will be always pending until an i/o event happens, it may happen that onClose() will be called by some thread scheduled before another thread which is willing to process a last write packet.

那还有其他IoPackets的Stream对象派生类(例如TcpClient)又如何呢?例如一个用于ConnectEx().

And what about those Stream objects derivatives (e.g. TcpClient) which also have other IoPackets? e.g. one for ConnectEx().

可能的解决方案是让AtomicInt m_total_pending_requests; <-，并且当该原子整数达到0时，完成线程将调用onClose();

A possible solution would be to have AtomicInt m_total_pending_requests; <- and when this atomic integer reaches 0, the completion thread calls onClose();

但是，这最后一个解决方案无效吗?为什么我要使用原子整数来完成诸如关闭连接并取消分配其结构之类的基本操作?

But, is this last solution inefficent? Why do I have use atomic integers to do such a basic thing like closing a connection and deallocating its structures?

也许我对带有那些Stream对象的IOCP的设计完全走错了路，但是我想这是抽象用于处理HANDLE的非常常用的设计，所以我想听听IOCP专家的意见. 我是否必须更改所有内容，否则Stream对象的这种设计可以非常可靠，安全且最重要，快速地工作?我之所以这样问，是因为进行诸如关闭+释放内存之类的基本操作很复杂.

Maybe I'm totally in the wrong path with design for IOCPs with those Stream objects, but i guess this is a pretty common used design for abstracting HANDLEs, so I wanted to hear some opinion from IOCPs gurus. Do I have to change everything, or this design of Stream objects could work pretty solid, safe, and most important, fast? I ask this because of the complexity came up for doing such a basic thing like close+free memory.

推荐答案

引用计数解决了该问题.

Reference counting solves the problem.

执行任何将生成IOCP补全的操作时，您只需增加一个参考计数，并在处理完补全后将其递减.尽管有一些未完成的操作，但您的引用大于1并且句柄对象有效.完成所有操作后，您的引用将降为0，并且该句柄可以自行删除.

You simply increment a reference count when you do anything which will generate an IOCP completion and decrement it once you have finished processing the completion. Whilst there are operations outstanding your reference is > 1 and the handle object is valid. Once all operations complete your reference drops to 0 and the handle can delete itself.

我对数据缓冲区做同样的事情，因为我的代码允许每个连接进行多个并发的发送/接收操作，因此不能将重叠的结构存储在句柄中.

I do the same for data buffers as my code allows for multiple concurrent send/recv operations to be in progress per connection and so storing overlapped structures in the handle itself isn't an option.

我有一些代码可以演示上述内容，您可以从这里.

I have some code which demonstrates the above, you can download it from here.

这篇关于I/O完成端口* LAST *称为回调，或:在安全的地方清理事物的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！