如果您的消费者少于分区，会发生什么? [英] If you have less consumers than partitions, what happens?

问题描述

如果您的消费者数量少于分区数量，这是否仅仅意味着您不会消费给定主题上的所有消息?

If you have less consumers than partitions, does that simply mean you will not consume all the messages on a given topic?

在云环境中，您打算如何跟踪正在运行的消费者数量以及指向给定主题的消费者数量#partition?

In a cloud environment, how are you suppose to keep track how many consumers are running and how many are pointing to a given topic#partition?

如果您在给定的主题#partition 上有多个消费者怎么办?我猜消费者必须以某种方式跟踪它已经处理的消息，以防出现重复?

What if you have multiple consumers on a given topic#partition? I guess the consumer has to somehow keep track of what messages it has already processed in case of duplicates?

推荐答案

其实每个消费者都属于一个消费群体.当Kafka集群向一个消费者组发送数据时，一个分区的所有记录都会发送给该组中的一个消费者.

In fact, each consumer belongs to a consumer group. When Kafka cluster sends data to a consumer group, all records of a partition will be sent to a single consumer in the group.

如果一个组中的分区比消费者多，那么一些消费者将消费来自多个分区的数据.如果组中的消费者多于分区，则某些消费者将无法获得数据.如果您向组中添加新的消费者实例，它们将接管旧成员的一些分区.如果您从组中删除一个消费者(或消费者死亡)，其分区将重新分配给其他成员.

If there're more paritions than consumers in a group, some consumers will consume data from more than one partition. If there're more consumers in a group than paritions, some consumers will get no data. If you add new consumer instances to the group, they will take over some partitons from old members. If you remove a consumer from the group (or the consumer dies), its partition will be reassigned to other member.

现在让我们来看看您的问题:

Now let's take a look at your questions:

如果您的消费者数量少于分区数量，这是否仅仅意味着您不会消费给定主题上的所有消息?

If you have less consumers than partitions, does that simply mean you will not consume all the messages on a given topic?

没有.同一个消费者组中的一些消费者会消费来自多个分区的数据.

NO. Some consumers in the same consumer group will consume data from more than one partition.

在云环境中，您打算如何跟踪正在运行的消费者数量以及指向给定主题的消费者数量#partition?

In a cloud environment, how are you suppose to keep track how many consumers are running and how many are pointing to a given topic#partition?

Kafka 会处理的.如果有新的消费者加入，或者老的消费者死亡，Kafka 会做 reblance.

Kafka will take care of it. If new consumers join the group, or old consumers dies, Kafka will do reblance.

如果您在给定主题#partition 上有多个消费者怎么办?

What if you have multiple consumers on a given topic#partition?

您不能有多个消费者(在一个消费者组中)来消费来自单个分区的数据.但是，如果有多个消费者组，则同一分区可以被每个消费者组中的一个(且只有一个)消费者消费.

You CANNOT have multiple consumers (in a consumer group) to consume data from a single parition. However, if there're more than one consumer group, the same partition can be consumed by one (and only one) consumer in each consumer group.

这篇关于如果您的消费者少于分区，会发生什么?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！