ZeroMQ 轮询线程安全 [英] ZeroMQ poll thread safety

问题描述

我有一个线程正在对 ZMQ 轮询器进行轮询:

I have a thread that is polling on a ZMQ Poller:

poller.poll(timeout)

该线程也是通过轮询器中注册的套接字接收和发送消息的线程.

This thread is also the one which receives and sends back messages over the sockets registered in the poller.

然后我有另一个线程，最终可能会创建一个新的套接字并注册它以轮询输入事件:

Then I have another thread that may, eventually, create a new socket and register it for polling on input events:

socket = context.socket(...)
socket.bind/connect(...)
poller.register(socket, zmq.POLLIN)

一旦socket被注册，后面的线程就不会再碰它了.

Once the socket is registered, the latter thread will not touch it again.

这样安全吗?

我得到的答案/评论是关于我不应该这样做的.或者哪些是指南的建议(我已经知道了).但这并不能真正回答我的问题.

The answers/comments I got were about how I should not be doing this. Or which are The Guide's recommendations (which I already knew). But that does not really answer my question.

更具体地说，我会说我正在使用用于 ZeroMQ 的 pyzmq Python 绑定.

To be more specific, I would say that I am working with pyzmq Python bindings for ZeroMQ.

现在，虽然 ZeroMQ 套接字不是线程安全的，但确实可以将它们从一个线程传输到另一个线程，只要在传输过程中有一个完整的内存屏障.

Now, although ZeroMQ sockets are not thread safe, it is indeed possible to transfer them from one thread to another as long as there is a full memory barrier during the transfer.

所以第一个问题是:我需要在那里设置一个显式的内存屏障吗?请注意，有一个线程创建并绑定/连接套接字，然后注册它，但它不会再次使用该线程.是否存在实际冲突?有没有可能我应该明确阻止两个线程访问套接字?

So the first question would be: do I need to set an explicit memory barrier in there? Note that there is one thread that creates and binds/connects the socket and then it registers it, but it will not be using that thread again. Is there an actual conflict? could there be a moment in which I should be explicitly preventing access to the socket from both threads?

那么第二个问题是:在轮询线程中注册套接字是否安全?大多数情况下，执行轮询的线程正忙于做其他事情，但也可能发生轮询等待超时的情况.在这种情况下，我是否需要使用锁来防止对轮询器的并发访问?或者在另一个线程轮询它时在轮询器中注册新套接字是否安全?

Then the second question would be: is registering a socket in a poller thread-safe? Most of the time the thread that performs the polling is busy doing other stuff, but it could happen that it is polling waiting for a timeout. In that case, do I need to use a lock to prevent concurrent access to the poller? or is it safe to register the new socket in the poller while the other thread is polling it?

我正在使用 Pyro4 来处理和配置远程进程(即:它们的 ZeroMQ 连接及其行为).可以使用 Pyro 代理 非常轻松地完成初始配置.然而，当我开始这个过程时，我实际上是用一个专用线程运行主循环(Pyro oneway call) 继续运行，但如果我再次使用 Pyro 代理访问该对象，则此访问来自另一个线程.

I am using Pyro4 to handle and configure remote processes (i.e.: their ZeroMQ connections and their behavior). The initial configuration can be done with the Pyro Proxy very esaily. However, when I start the process, I am in fact running the main loop with a dedicated thread (Pyro oneway call) that keeps running, but if I access the object with the Pyro Proxy again, then this access is from another thread.

所以这个想法是为了避免修改远程对象的类，但仍然允许使用 Pyro 来配置远程对象，即使它们正在运行.只要新套接字的创建+绑定/连接+注册对另一个线程来说是安全的，我就很好.

So the idea is to avoid modifying the remote object's class but still allow the use of Pyro for configuring the remote objects even when they are running. As long as the creation + binding/connecting + registering of new sockets is safe from another thread, I am good.

推荐答案

一旦socket被注册，后面的线程就不会再碰它了.这样安全吗?

Once the socket is registered, the latter thread will not touch it again. Is this safe?

没有

不仅需要安全解决方案，而且还需要向供应商方面实际证明稳定和有保证的系统行为的行业(无论是由于明智的祖父、对 QA/TQM 的深信还是由于强加的法规)MIL/GOV/航空航天/医疗保健/制药/汽车等细分市场供应商管理)将直接拒绝.

Industries that not only require safe solutions, but also export the responsibility to actually prove both the stable and warranted system behaviour to the vendor side (be it due to wise grandfathers, a deep belief in QA/TQM or due to regulations imposed on MIL/GOV/aerospace/healthcare/pharma/automotive et al segment vendor management) would simply straight reject.

为什么?

……不会再碰它了."只是一个承诺.

" ... will not touch it again." is just a promise.

经过安全交叉验证的系统设计不会以少于避免碰撞的证据来解决.

Safety cross-validated system design does not settle with less than a proof of a collision avoidance.

让我引用 Pieter HINTJENS 的一本可爱的书代码连接，第 1 卷"——ZeroMQ 的必读作品:

Let me cite from a lovely book from Pieter HINTJENS "Code Connected, Vol.1" - a must read piece for ZeroMQ:

一些广泛使用的模型，尽管是整个行业的基础，但从根本上被打破了，共享状态并发就是其中之一.想要无限扩展的代码会像互联网一样，通过发送消息和共享任何内容，除了对损坏的编程模型的普遍蔑视.

您应该遵循一些规则来使用 ØMQ 编写愉快的多线程代码:

• 在其线程内私下隔离数据，永远不要在多个线程中共享数据.唯一的例外是 ØMQ 上下文，它们是线程安全的.
• 远离经典的并发机制，如互斥体、临界区、信号量等.这些是 ØMQ 应用程序中的反模式.
• 在您的进程开始时创建一个 ØMQ 上下文，并将其传递给您想要通过 inproc 套接字连接的所有线程.
• 使用附加线程在您的应用程序中创建结构，并使用PAIR 套接字通过inproc 将这些结构连接到它们的父线程.模式是:绑定父套接字，然后创建连接其套接字的子线程.
• 使用分离的线程来模拟具有自己上下文的独立任务.通过 tcp 连接这些.稍后您可以将这些移动到独立进程中，而无需显着更改代码.
• 线程之间的所有交互都作为 ØMQ 消息发生，您可以或多或少地正式定义.
• 不要在线程之间共享 ØMQ 套接字.ØMQ 套接字不是线程安全的. 从技术上讲，可以将套接字从一个线程迁移到另一个线程，但这需要技巧.唯一可以在线程之间共享套接字的地方是在需要对套接字进行垃圾收集之类的魔术的语言绑定中.

例如，如果您需要在一个应用程序中启动多个代理，您将希望在各自的线程中运行每个代理.在一个线程中创建代理前端和后端套接字，然后将套接字传递给另一个线程中的代理时，很容易出错.这可能一开始看起来可行，但在实际使用中会随机失败.请记住:不要使用或关闭套接字，除非在创建它们的线程中.

如果您遵循这些规则，您可以很容易地构建优雅的多线程应用程序，然后根据需要将线程拆分为单独的进程.应用程序逻辑可以位于线程、进程或节点中:无论您的规模需要什么.

ØMQ 使用原生操作系统线程而不是虚拟的绿色"线程.优点是您不需要学习任何新的线程 API，并且 ØMQ 线程可以干净地映射到您的操作系统.您可以使用英特尔的 ThreadChecker 等标准工具来查看您的应用程序在做什么.缺点是原生线程 API 并不总是可移植的，而且如果您拥有大量线程(数千个)，某些操作系统会承受压力.

Some widely used models, despite being the basis for entire industries, are fundamentally broken, and shared state concurrency is one of them. Code that wants to scale without limit does it like the Internet does, by sending messages and sharing nothing except a common contempt for broken programming models.

You should follow some rules to write happy multithreaded code with ØMQ:

• Isolate data privately within its thread and never share data in multiple threads. The only exception to this are ØMQ contexts, which are threadsafe.
• Stay away from the classic concurrency mechanisms like as mutexes, critical sections, semaphores, etc. These are an anti-pattern in ØMQ applications.
• Create one ØMQ context at the start of your process, and pass that to all threads that you want to connect via inproc sockets.
• Use attached threads to create structure within your application, and connect these to their parent threads using PAIR sockets over inproc. The pattern is: bind parent socket, then create child thread which connects its socket.
• Use detached threads to simulate independent tasks, with their own contexts. Connect these over tcp. Later you can move these to stand-alone processes without changing the code significantly.
• All interaction between threads happens as ØMQ messages, which you can define more or less formally.
• Don’t share ØMQ sockets between threads. ØMQ sockets are not threadsafe. Technically it’s possible to migrate a socket from one thread to another but it demands skill. The only place where it’s remotely sane to share sockets between threads are in language bindings that need to do magic like garbage collection on sockets.

If you need to start more than one proxy in an application, for example, you will want to run each in their own thread. It is easy to make the error of creating the proxy frontend and backend sockets in one thread, and then passing the sockets to the proxy in another thread. This may appear to work at first but will fail randomly in real use. Remember: Do not use or close sockets except in the thread that created them.

If you follow these rules, you can quite easily build elegant multithreaded applications, and later split off threads into separate processes as you need to. Application logic can sit in threads, processes, or nodes: whatever your scale needs.

ØMQ uses native OS threads rather than virtual "green" threads. The advantage is that you don’t need to learn any new threading API, and that ØMQ threads map cleanly to your operating system. You can use standard tools like Intel’s ThreadChecker to see what your application is doing. The disadvantages are that native threading APIs are not always portable, and that if you have a huge number of threads (in the thousands), some operating systems will get stressed.

<小时>

如果您要跨线程共享套接字，不要.它会导致随机怪异和崩溃.

---

If you’re sharing sockets across threads, don’t. It will lead to random weirdness, and crashes.

我们可以假设轻"条件:系统没有压力，从未达到高水位线，没有大拥塞.只有一个线程运行应用程序(轮询和执行输入的任务).所以大部分时间(99.99%)没有并发.现在，只有当第二个线程出现只是为了将套接字添加到池中时才会发生并发.永远不会有超过 2 个线程被执行.并且第二个线程将始终被限制向池中添加新套接字(一旦添加，套接字将转移到主线程).这足以满足边界条件吗?– Peque

We could assume "light" conditions: system not stressed, high-watermark never reached, no big congestions. There is just a single thread running the application (polling and executing tasks on input). So most of the time (99.99%) there is no concurrency. Now, concurrency only occurs when a second thread appears just to add a socket to the pool. There will never be more than 2 threads being executed. And the second thread will be always restricted to adding new sockets to the pool (once added the socket is transferred to the main thread). Is this enough for boundary conditions? – Peque

update-II中增加了更多的原理图用例细节，专业的解决方案不会浪费时间，并通过使用线程干净的设计避免任何隐藏的风险.

The more the schematic use-case details were added in update-II, the professional solution shall not lose time and shall avoid any hidden risks by using thread-clean design.

#T1 a poller-maintainer -has Context() instance control
                        -has graceful .close() + .term() responsibility
                        -has POLLER instance under it's own control
                        -has PAIR  .bind(    "inproc://worker2poller" )
                        -has PAIR  .recv() <add_socket>-request processing responsibility

#T2 a worker-process:   -has PAIR  .connect( "inproc://worker2poller" )
                        -has PAIR  .send() privilege to ask T1 to add a socket & include it into POLLER

<小时>

虽然 GIL 无论如何避免了任何机会找到运行 PARALLEL 的 python 线程，但纯OOP 设计的动机是让架构保持干净和分离的职责，并保持正式通信模式完全可扩展.

---

While GIL anyway avoids any chance to find the python threads run PARALLEL, the pure OOP-design is the motivation to keep the architecture with both clean and separated responsibilities and keeping the Formal Communication Patterns fully scaleable.

这篇关于ZeroMQ 轮询线程安全的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！