如果您的消费者少于分区，那么会发生什么? [英] If you have less consumers than partitions, what happens?

问题描述

如果您的使用者少于分区，那么这是否就意味着您不会使用给定主题上的所有消息?

If you have less consumers than partitions, does that simply mean you will not consume all the messages on a given topic?

在云环境中，您应该如何跟踪正在运行的消费者数量以及指向给定主题#partition的消费者数量?

In a cloud environment, how are you suppose to keep track how many consumers are running and how many are pointing to a given topic#partition?

如果给定topic#partition上有多个使用者，该怎么办?我想消费者必须以某种方式跟踪重复的情况下已经处理了哪些消息?

What if you have multiple consumers on a given topic#partition? I guess the consumer has to somehow keep track of what messages it has already processed in case of duplicates?

推荐答案

实际上，每个消费者都属于消费者群体.当Kafka群集将数据发送到使用者组时，分区的所有记录将被发送到组中的单个使用者.

In fact, each consumer belongs to a consumer group. When Kafka cluster sends data to a consumer group, all records of a partition will be sent to a single consumer in the group.

如果一个组中的分区数量超过了消费者，则某些消费者将使用多个分区中的数据.如果组中的消费者多于分区，则某些消费者将不会获得任何数据.如果将新的使用者实例添加到组中，它们将接管旧成员中的一些参与者.如果您从组中删除某个消费者(或该消费者死亡)，则其分区将重新分配给其他成员.

If there're more paritions than consumers in a group, some consumers will consume data from more than one partition. If there're more consumers in a group than paritions, some consumers will get no data. If you add new consumer instances to the group, they will take over some partitons from old members. If you remove a consumer from the group (or the consumer dies), its partition will be reassigned to other member.

现在让我们看看您的问题:

Now let's take a look at your questions:

如果您的使用者少于分区，那么这是否就意味着您不会使用给定主题上的所有消息?

If you have less consumers than partitions, does that simply mean you will not consume all the messages on a given topic?

否.同一使用者组中的某些使用者将使用多个分区中的数据.

NO. Some consumers in the same consumer group will consume data from more than one partition.

在云环境中，您应该如何跟踪正在运行的消费者数量以及指向给定主题#partition的消费者数量?

In a cloud environment, how are you suppose to keep track how many consumers are running and how many are pointing to a given topic#partition?

Kafka会照顾好它.如果新消费者加入该小组，或者老消费者死了，那么卡夫卡一定会感到厌恶.

Kafka will take care of it. If new consumers join the group, or old consumers dies, Kafka will do reblance.

如果给定主题#partition上有多个使用者，该怎么办?

What if you have multiple consumers on a given topic#partition?

您不能有多个使用者(在使用者组中)来使用单个分区中的数据.但是，如果有一个以上的消费者组，则同一分区可以由每个消费者组中的一个(并且只有一个)消费者使用.

You CANNOT have multiple consumers (in a consumer group) to consume data from a single parition. However, if there're more than one consumer group, the same partition can be consumed by one (and only one) consumer in each consumer group.

这篇关于如果您的消费者少于分区，那么会发生什么?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！