Kafka多个使用者用于一个分区 [英] Kafka multiple consumers for a partition

问题描述

我有一个生产者，可以将消息写入主题/分区.为了保持顺序，我想使用单个分区，我希望12个使用者从该单个分区读取所有消息(没有使用者组，所有消息应发给所有使用者).这是可以实现的吗?我读过一些论坛，每个分区只能读取一个用户.

I have a producer which writes messages to a topic/partition. To maintain ordering, i would like to go with single partition and I want 12 consumers to read all the messages from this single partition(no consumer group, all the messages should go to all consumers). Is this achievable? I read some forums that only one consumer can read per partition.

推荐答案

您可以使用SimpleConsumer来实现您要查询的内容-没有使用者组，所有使用者都可以读取一个分区.但是，这种方法意味着您必须自己处理偏移量存储和代理失败.

You may use SimpleConsumer to achieve exactly what you are asking - no consumer groups, all consumers can read a single partition. However this approach means you have to handle offset storing and broker failure handling yourself.

另一种选择是将高级使用者与不同的使用者组一起使用(您可以将随机UUID分配给每个使用者).这样，您将能够与所有使用者一起使用一个主题/分区，并且能够提交补偿并处理代理中断.

Another option is to use high level consumer with different consumer groups (you could just assign a random UUID to each consumer). This way you'll be able to consume one topic/partition with all consumers and be able to commit offsets and handle broker outage.

只有一个消费者可以消费一个主题/分区"规则仅适用于消费者组，例如IN GROUP中只有一个消费者可以同时消费一个主题/分区.

The rule "only a single consumer can consume a topic/partition" applies only to consumer groups, e.g. only one consumer IN GROUP can consume one topic/partition simultaneously.

这篇关于Kafka多个使用者用于一个分区的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！