ZeroMQ PUB 套接字在连接时缓冲我所有的输出数据 [英] ZeroMQ PUB socket buffers all my out going data when it is connecting

问题描述

我注意到一个 zeromq PUB 套接字将缓冲所有传出数据，例如，如果它正在连接

I noticed that a zeromq PUB socket will buffers all outgoing data if it is connecting, for example

import zmq
import time
context = zmq.Context()

# create a PUB socket
pub = context.socket (zmq.PUB)
pub.connect("tcp://127.0.0.1:5566")
# push some message before connected
# they should be dropped
for i in range(5):
    pub.send('a message should not be dropped')

time.sleep(1)

# create a SUB socket
sub = context.socket (zmq.SUB)
sub.bind("tcp://127.0.0.1:5566")
sub.setsockopt(zmq.SUBSCRIBE, "")

time.sleep(1)

# this is the only message we should see in SUB
pub.send('hi')

while True:
    print sub.recv()

子绑定在这些消息之后，它们应该被丢弃，因为如果没有人连接到 PUB，它应该丢弃消息.但它不会丢弃消息，而是缓冲所有消息.

The sub binds after those messages, they should be dropped, because PUB should drop messages if no one connected to it. But instead of dropping messages, it buffers all messages.

a message should not be dropped
a message should not be dropped
a message should not be dropped
a message should not be dropped
a message should not be dropped
hi

如您所见，那些不应丢弃的消息"由套接字缓冲，一旦连接，它就会将它们刷新到 SUB 套接字.如果我在 PUB 套接字上绑定，并在 SUB 套接字上连接，则它可以正常工作.

As you can see, those "a message should not be dropped" are buffered by the socket, once it gets connected, it flush them to SUB socket. If I bind at the PUB socket, and connect at the SUB socket, then it works correctly.

import zmq
import time
context = zmq.Context()

# create a PUB socket
pub = context.socket (zmq.PUB)
pub.bind("tcp://127.0.0.1:5566")
# push some message before connected
# they should be dropped
for i in range(5):
    pub.send('a message should not be dropped')

time.sleep(1)

# create a SUB socket
sub = context.socket (zmq.SUB)
sub.connect("tcp://127.0.0.1:5566")
sub.setsockopt(zmq.SUBSCRIBE, "")

time.sleep(1)

# this is the only message we should see in SUB
pub.send('hi')

while True:
    print repr(sub.recv())

而且你只能看到输出

'hi'

这种奇怪的行为导致了一个问题，它将所有数据缓存在一个连接的套接字上，我有两台服务器，服务器 A 将数据发布到服务器 B

This kind of strange behavior cause a problem, it buffers all data on a connecting socket, I have two servers, server A publishes data to server B

Server A -- publish --> Server B

如果服务器 B 上线，它工作正常.但是如果我启动了服务器 A 而没有启动服务器 B 呢?

It works fine if server B gets online. But what if I start the Server A and do not start Server B?

结果，服务器 A 上连接的 PUB 套接字保留了所有这些数据，内存使用率越来越高.

As the result, the connecting PUB socket on Server A keeps all those data, the memory usage gets higher and higher.

问题来了，这种行为是错误还是功能?如果是功能，我在哪里可以找到提到此行为的文档?以及如何停止连接的 PUB 套接字缓冲所有数据?

Here is the problem, is this kind of behavior a bug or feature? If it is feature, where can I find a document that mentions this behavior? And how can I stop the connecting PUB socket buffers all data?

谢谢.

推荐答案

socket 是阻塞还是丢弃消息取决于 ZMQ::Socket 文档(下面的重点是我的):

Whether the socket blocks or drops messages depends on the socket type as described in the ZMQ::Socket documentation (emphasis below is mine):

ZMQ::HWM:检索高水位

ZMQ::HWM: Retrieve high water mark

ZMQ::HWM 选项应检索高水位标记指定的套接字.高水位线是对最大值的硬限制0MQ 应在内存中排队等待任何未完成消息的数量指定套接字正在与之通信的单个对等方.

The ZMQ::HWM option shall retrieve the high water mark for the specified socket. The high water mark is a hard limit on the maximum number of outstanding messages 0MQ shall queue in memory for any single peer that the specified socket is communicating with.

如果达到此限制，套接字将进入异常状态并根据套接字类型，0MQ 应采取适当的阻止或丢弃已发送邮件等操作.请参阅ZMQ::Socket 中的各个套接字描述以获取有关确切的详细信息对每种套接字类型采取的操作.

If this limit has been reached the socket shall enter an exceptional state and depending on the socket type, 0MQ shall take appropriate action such as blocking or dropping sent messages. Refer to the individual socket descriptions in ZMQ::Socket for details on the exact action taken for each socket type.

默认的 ZMQ::HWM 值为零意味着无限制".

The default ZMQ::HWM value of zero means "no limit".

您可以通过查看 ZMQ::HWM 选项操作 的套接字类型文档来查看它是否会阻塞或丢弃，该套接字类型将是 Block 或 >放下.

You can see if it will block or drop by looking through the documentation for the socket type for ZMQ::HWM option action which will either be Block or Drop.

ZMQ::PUB 的动作是 Drop，所以如果它没有掉线，你应该检查 HWM(高水位线)值并注意 Dropstrong>默认的 ZMQ::HWM 值为零意味着无限制"，意味着它不会进入异常状态，直到系统内存耗尽(此时我不知道它的行为如何).

The action for ZMQ::PUB is Drop, so if it is not dropping you should check the HWM (High Water Mark) value and heed the warning that The default ZMQ::HWM value of zero means "no limit", meaning that it will not enter an exceptional state until the system runs out of memory (at which point I don't know how it behaves).

这篇关于ZeroMQ PUB 套接字在连接时缓冲我所有的输出数据的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！