删除Zookeeper中的kafka消费者组 [英] removing a kafka consumer group in zookeeper

问题描述

我在使用Zookeeper 3.4.6的kafka_2.9.2-0.8.1.1.

I'm using kafka_2.9.2-0.8.1.1 with zookeeper 3.4.6.

是否存在可以从Zookeeper中自动删除使用者组的实用程序?还是可以只删除zookeeper中/consumers/[group_id]下的所有内容?如果是后者，我还有什么想念的吗?可以通过实时系统完成吗?

Is there a utility that can automatically remove a consumer group from zookeeper? Or can I just remove everything under /consumers/[group_id] in zookeeper? If the latter, is there anything else I'm missing & can this be done with a live system?

更新: 从kafka 2.3.0版开始，有一个新实用程序:

Update: As of kafka version 2.3.0, there is a new utility:

> bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --delete --group my-group

> bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --delete --group my-group

相关文档: http://kafka.apache.org/documentation/#basic_ops_consumer_lag

有关更多讨论，请参见下文

See below for more discussion

推荐答案

目前，据我所知，删除Kafka用户组的唯一方法是手动删除Zookeeper路径/consumers/[group_id].

Currently, as I know, the only way to remove a Kafka consumer group is manually deleting Zookeeper path /consumers/[group_id].

如果您只想删除使用者组，则无需担心手动删除Zookeeper路径，但是如果您使用它来倒回偏移量，则下面的内容会有所帮助.

If you just want to delete a consumer group, there is nothing to worry about manually deleting the Zookeeper path, but if you do it for rewinding offsets, the below will be helpful.

首先，您应该在删除Zookeeper路径之前停止所有属于使用者组的使用者.如果您不这样做，这些使用者将不会使用新产生的消息，并且很快将关闭与Zookeeper群集的连接.

First of all, you should stop all the consumers belongs to the consumer group before removing the Zookeeper path. If you don't, those consumers will not consume newly produced messages and will soon close connections to the Zookeeper cluster.

重新启动使用者时，如果希望使用者从头开始，请将auto.offset.reset属性赋予smallest(或在新的Kafka版本中为earliest).该属性的默认值为largest(在新的Kafka版本中为latest)，这使您的重启使用者在最大偏移量之后读取，而偏移量仅消耗新生成的消息.有关该属性的更多信息，请参阅Kafka文档中的消费者配置.

When you restart the consumers, if you want the consumers to start off from the beginning, give auto.offset.reset property to smallest (or earliest in new Kafka releases). The default value of the property is largest (or latest in new Kafka releases) which makes your restarting consumers read after the largest offset which in turn consuming only newly produced messages. For more information about the property, refer to Consumer Config in the Kafka documentation.

仅供参考，有一个问题如何在消费者中回退偏移量?，在Kafka常见问题解答中，但这并没有给我太大帮助.

FYI, there is a question How can I rewind the offset in the consumer? in Kafka FAQ, but it gave me not much help.

这篇关于删除Zookeeper中的kafka消费者组的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！