删除zookeeper中的kafka消费者组 [英] removing a kafka consumer group in zookeeper

问题描述

我使用 kafka_2.9.2-0.8.1.1 和 zookeeper 3.4.6.

I'm using kafka_2.9.2-0.8.1.1 with zookeeper 3.4.6.

有没有可以自动从zookeeper中删除消费者组的实用程序?或者我可以删除zookeeper中/consumers/[group_id]下的所有内容吗?如果是后者，还有什么我想念的吗?这可以通过实时系统完成吗?

Is there a utility that can automatically remove a consumer group from zookeeper? Or can I just remove everything under /consumers/[group_id] in zookeeper? If the latter, is there anything else I'm missing & can this be done with a live system?

更新:从 kafka 2.3.0 版开始，有一个新的实用程序:

Update: As of kafka version 2.3.0, there is a new utility:

<代码>>bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --delete --group my-group

相关文档:http://kafka.apache.org/documentation/#basic_ops_consumer_lag

更多讨论见下文

推荐答案

目前，据我所知，删除 Kafka 消费者组的唯一方法是手动删除 Zookeeper 路径 /consumers/[group_id].

Currently, as I know, the only way to remove a Kafka consumer group is manually deleting Zookeeper path /consumers/[group_id].

如果你只是想删除一个消费者组，手动删除Zookeeper路径没有什么可担心的，但是如果你是为了rewinding offsets而做的，下面的会很有帮助.

If you just want to delete a consumer group, there is nothing to worry about manually deleting the Zookeeper path, but if you do it for rewinding offsets, the below will be helpful.

首先，在移除Zookeeper路径之前，您应该停止所有属于该消费者组的消费者.如果不这样做，这些消费者将不会使用新生成的消息，并且很快就会关闭与 Zookeeper 集群的连接.

First of all, you should stop all the consumers belongs to the consumer group before removing the Zookeeper path. If you don't, those consumers will not consume newly produced messages and will soon close connections to the Zookeeper cluster.

当您重新启动消费者时，如果您希望消费者从头开始，请将 auto.offset.reset 属性赋予 smallest(或 earliest 在新的 Kafka 版本中).该属性的默认值是 largest(或新 Kafka 版本中的 latest)，这使您重新启动的消费者在最大偏移后读取，而这反过来又只消耗新生成的消息.有关该属性的更多信息，请参阅 Kafka 文档中的 消费者配置.

When you restart the consumers, if you want the consumers to start off from the beginning, give auto.offset.reset property to smallest (or earliest in new Kafka releases). The default value of the property is largest (or latest in new Kafka releases) which makes your restarting consumers read after the largest offset which in turn consuming only newly produced messages. For more information about the property, refer to Consumer Config in the Kafka documentation.

仅供参考，有一个问题如何在消费者中回滚偏移量? 在 Kafka FAQ 中，但它给我的帮助不大.

FYI, there is a question How can I rewind the offset in the consumer? in Kafka FAQ, but it gave me not much help.

这篇关于删除zookeeper中的kafka消费者组的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！