Celery 广播与 RabbitMQ 扇出 [英] Celery broadcast vs RabbitMQ fanout

问题描述

我最近一直在使用 Celery，但我不喜欢它.它的配置凌乱、过于复杂且文档不全.

我想使用 Celery 从单个生产者向多个消费者发送广播消息.令我困惑的是 Celery 术语与底层传输 RabbitMQ 术语之间的差异.

在 RabbitMQ 中，您可以有一个 ，向多个消费者发送消息:

我什至不明白，Celery 广播队列应该如何工作，因为具有多个消费者的 RabbitMQ 队列用于负载平衡.所以在RabbitMQ中如果多个消费者(即消费者池)连接到同一个队列，只有一个消费者会接收和处理消息，在 RabbitMQ 文档中称为循环法.

另外，Celery 关于广播的文档确实不够.我应该为广播队列指定哪种类型的 RabbitMQ 交换，扇出与否?你能提供一个完整的例子吗?

所以，我要求的是 (1) 澄清 Celery 中广播队列的概念和实现，以及 (2) 广播队列配置的完整示例.谢谢.

解决方案

这有帮助吗?
http://celery.readthedocs.org/en/latest/userguide/routing.html#exchanges-queues-and-routing-keys

似乎 Celery 中的 'queue' 定义包含交换，因此您可以在 Exchange('fanout') 交换类型之上定义一个 Celery 队列，它将具有多个的底层实现RabbitMQ 队列.

在这种情况下，我猜您不希望 Celery 配置中有广播"队列，除非您真的希望多个工作人员处理同一任务.

I've been working with Celery lately and I don't like it. It's configuration is messy, overcomplicated and poorly documented.

I want to send broadcast messages with Celery from a single producer to multiple consumers. What confuses me is discrepancy between Celery terms and terms of underlying transport RabbitMQ.

In RabbitMQ you can have a single fanout Exchange and multiple Queues to broadcast messages:

But in Celery the terms are all messed up: here you can have a broadcast Queue, which sends messages to multiple consumers:

I don't even understand, how Celery broadcast queue is supposed to work at all, cause RabbitMQ queues with multiple consumers are meant for load balancing. So in RabbitMQ if multiple consumers (i.e. a pool of consumers) are connected to the same queue, only one consumer will receive and process message, which is called round robin in RabbitMQ docs.

Also, Celery documentation on broadcast is really insufficient. What type of RabbitMQ exchange should I specify for Broadcast queue, fanout or not? Could you supply a full example?

So, what I'm asking for is (1) clarification of concept and implementation of Broadcast queues in Celery and (2) a complete example of Broadcast queues configuration. Thank you.

解决方案

Does this help?
http://celery.readthedocs.org/en/latest/userguide/routing.html#exchanges-queues-and-routing-keys

It appears the 'queue' definition in Celery includes the exchange, so you can define a Celery queue on top of Exchange('fanout') exchange type, which will have an underlying implementation of multiple RabbitMQ queues.

In this case I would guess you don't want a 'broadcast' queue in the Celery config, unless you really want multiple workers processing the same task.

这篇关于Celery 广播与 RabbitMQ 扇出的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！