相对于使用 ZeroMQ 的一个订阅者，向没有订阅者发布是否很慢? [英] Is it slow to publish to no subscribers relative to one subscriber with ZeroMQ?

问题描述

More Detail:

我们正在编写一个 ZeroMQ 应用程序，其中速度非常重要.

我们有许多节点通过 进行通信REQ/REP 以及 PUB/SUB 和网络自动选择 { ipc: |tcp: } 传输类，如果节点在同一台机器上.

We're writing a ZeroMQ application, where speed is very important.

We have many nodes that communicate via REQ/REP as well as PUB/SUB and the network automatically selects { ipc: | tcp: } transport-class, if the nodes are on the same machine.

我们有时希望记录某些节点之间的消息.使用 PUB/SUB 这很容易，我们只需要一个订阅发布者的日志节点".但是，使用REQ/REP，我们无法在不成为代理或以其他方式减慢连接速度的情况下读取请求/响应.

We'd like to sometimes log the messages between certain nodes. With PUB/SUB this is easy, we just have a "logging node" subscribed to the publisher. However, with REQ/REP, we cannot read the request/response without becoming a proxy or otherwise slowing down the connection.

我们正在考虑让所有使用 REQ/REP 的节点在每次发送消息时都发布到一个唯一的 TCP 地址(因此每个节点都有一个日志地址"，它们发送所有消息到 )，然后我们将订阅我们感兴趣的记录地址"，如果我们想登录.

We're considering having all of the nodes using REQ/REP publish to a unique TCP address every time they send a message ( so each node has a "logging address" that they send all their messages to ), we'll then just subscribe to the "logging addresses" we're interested in, if we want to log.

Question:

如果我们没有订阅日志地址"，我们会遭受性能损失吗?日志记录过程中速度变慢是可以的，但正常操作期间的性能损失是不可取的.

Will we suffer a performance penalty if we ARE NOT subscribed to the "logging address"? A slowdown during logging is okay, but performance penalties during normal operation are not desirable.

推荐答案

订阅如何运作?

在 v3.1 之前，订阅机制(又名 TOPIC 过滤器)是在 SUB 上处理的-side，所以这部分 processign 分布在所有 SUB-s 中(以跨所有涉及的传输类的统一宽数据流量为代价)并且没有惩罚，除了采购PUB 端的此类数据流相关工作负载(参考下文).

How the subscription works?

Until v3.1, subscription mechanics ( a.k.a. a TOPIC-filter ) was handled on the SUB-side, so this part of the processign got distributed among all SUB-s ( at a cost of uniformly wide data-traffic across all transport-classes involved ) and there was no penalty, except for a sourcing such data-flow related workload ( ref. below ) on the PUB-side.

自 v3.1 起，TOPIC 过滤器在 PUB 端处理，在这种处理开销的成本，但节省了所有以前浪费的传输容量，消耗只是为了稍后在 SUB 端实现消息与 TOPIC 过滤器不匹配，将被处理掉.

Since v3.1, the TOPIC-filter is processed on the PUB-side, at a cost of such a processing overhead, but saving all the previously wasted transport-capacities, consumed just to later realise at the SUB-side the message is not matching the TOPIC-filter and will be disposed off.

正如问题中所假设的，比较应该与:
场景A:一个PUB-process没有SUB-消费者连接/订阅任何 TOPIC-filter
Scenario B:PUB-process 有一个 SUB-消费者连接/订阅了一个主题过滤器

As postulated in Question, the comparison ought be related to:
Scenario A: a PUB-process has no SUB-consumer connected/subscribed to any TOPIC-filter
Scenario B: a PUB-process has one SUB-consumer connected/subscribed to a TOPIC-filter

ZeroMQ 具有全状态内部 FSA，既节省了编程架构，又节省了资源利用率.这就是说，场景 A 产生零工作负载，即没有与 PUB 处理相关的影响，因为在第一个真正的 SUB-连接.

ZeroMQ has a state-full internal FSA, which saves both the programming architecture and resouces utilisation. This said, the Scenario A produces zero workload, i.e. has no impact related to PUB-processing, as there none such processing actually happens until a first real SUB-connects.

如果您的 Scenario B 确实代表了用例，则与仅服务一个 SUB 消费者相关的额外处理开销很容易衡量:

If your Scenario B does indeed represent the use-case, the additional processing overhead, related to serving just one single SUB-consumer, is easily measureable:

from zmq import Stopwatch as StopWATCH
aStopWATCH = StopWATCH()
# -----------------------------------------------------------------<TEST_SECTION>-start
aStopWATCH.start();s_PUB_send.send( "This is a MESSAGE measured for 0 SUB-s", zmq.NOBLOCK );t0 = aStopWATCH.stop()
# -----------------------------------------------------------------<TEST_SECTION>-end


# .connect the first SUB-process and let it .setsockopt() for v3.1+ accordingly


# -----------------------------------------------------------------<TEST_SECTION>-start
aStopWATCH.start();s_PUB_send.send( "This is a MESSAGE measured for 1 SUB-s", zmq.NOBLOCK );t1 = aStopWATCH.stop()
# -----------------------------------------------------------------<TEST_SECTION>-end

print "\nZeroMQ has consumed {0:} [us] for PUB-side processing on [Scenario A]\nZeroMQ has consumed {1:} [us] for PUB-side processing on [Scenario B]".format( t0, t1 )

如果 .connect()-ed(FSA 知道实时交易对手)，但未订阅任何内容(.setsockopt( "" ) ) SUB-消费者处理是要验证的，不管实际使用的{ pre-v3.1 |v3.1+ }-API(在分布式系统中处理不同版本的 API 时要小心，在这种情况下，无法为远程节点强制执行统一的 API 版本，这是一个人无法控制的配置管理).

The same test might be re-used to measure such difference in case .connect()-ed ( FSA-knows about the live counterparty ), but subscribed to nothing ( .setsockopt( "" ) ) SUB-consumer processing is to be validated, irrespective of the actually used { pre-v3.1 | v3.1+ }-API ( just be carefull to handle different versions of API in distributed-systems, where one cannot enforce uniform API-versions for remote nodes, that are outside of one's control of the Configuration Management ).

可以进一步微调性能已经受到限制的项目的性能属性.

One may further fine tune performance attributes for performance already constrained projects.

对于选定的处理任务、性能，人们可能会猜测这里的先验不是那么难，可以通过将每个工作流的处理映射到多个创建的 I/O 线程的分离子集来分离工作负载流的处理:

For selected processing tasks, performance, that one may guess a-priori is not so tough here, one may segregate workload-streams' processing by mapping each one on disjunct sub-sets of the multiple created I/O-threads:

map s_REQ_sock.setsockopt( ZMQ_AFFINITY, 0 );
和 s_PUB_send.setsockopt( ZMQ_AFFINITY, 1 );
分别s_SUB_recv.setsockopt( ZMQ_AFFINITY, ... );

set s_SUB_recv.setsockopt( ZMQ_MAXMSGSIZE, 32000 ); // protective ceiling
set s_SUB_recv.setsockopt( ZMQ_CONFLATE,   True );  // retain just the last msg
set s_SUB_recv.setsockopt( ZMQ_LINGER,     0 );     // avoid blocking
set s_SUB_recv.setsockopt( ZMQ_TOS,        anAppToS_NETWORK_PRIO_CODE );

这篇关于相对于使用 ZeroMQ 的一个订阅者，向没有订阅者发布是否很慢?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！