即使没有消费者，消费者群体仍陷于“重新平衡" [英] Consumer group stuck in 'rebalancing' even though there are no consumers
问题描述
我正在使用 kafka 版本 2.4.1(最近从 2.2.0 升级到 2.4.1)并注意到一个奇怪的问题.
I am using kafka version 2.4.1(recently upgraded to 2.4.1 from 2.2.0) and noticed a strange problem.
即使应用程序(kafka 流)已关闭(没有正在运行的应用程序)，但消费者组命令将状态返回为重新平衡.我们的应用程序作为 kubernetes pod 运行.
Even though application(kafka streams) is down (there is no application which is running ) but the consumer group command returns the state as rebalancing. Our application runs as kubernetes pod.
root@bastion-0:# ./kafka-consumer-groups --describe --group groupname --bootstrap-server kafka-0.local:9094
Warning: Consumer group 'groupname' is rebalancing.
我已经等了一段时间(30 分钟)，即使应用程序关闭，命令仍然报告重新平衡".
I have waited for some amount of time now(30 mins) and still the command reports 'rebalancing' even though application is down.
即使我尝试删除该组，它也会显示以下消息.
Even if i try to delete the group, it gives the following message.
root@bastion-0:/app/kafka_2.12-2.4.1/bin# ./kafka-consumer-groups.sh --delete --group group1 --bootstrap-server kafka.local:9094
Error: Deletion of some consumer groups failed:
* Group 'group1' could not be deleted due to: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.GroupNotEmptyException: The group is not empty.
root@bastion-0:/app/kafka_2.12-2.4.1/bin# ./kafka-consumer-groups.sh --delete --group group2 --bootstrap-server kafka.local:9094
Error: Deletion of some consumer groups failed:
* Group 'group2' could not be deleted due to: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.GroupNotEmptyException: The group is not empty.
当我查看组成员时，即使应用程序未运行，也会列出成员.是因为新的重新平衡协议(合作重新平衡)吗?
When i look at the group members, there are members listed even though application is NOT running. Is it because of new rebalance protocol(cooperative rebalance) ?
./kafka-consumer-groups 从哪里读取组成员信息.应用宕机是否保存会员信息?
From where does ./kafka-consumer-groups reads the group membership information. Does it save the member information if the application is down ?
更新:
我用不同的组名启动了应用程序，结果很好.我也可以描述这个组.即便如此，旧集团仍处于重新平衡"状态.
I brought up the application with a different group name and it came up fine. I can describe the group also. Even then the old group is in 'rebalancing' state.
新更新此外，我发现所有组的组协调器是 kafka 集群中的节点之一，当我重新启动该节点时，问题就消失了.
New Update Also, i found that group coordinator for all the groups was one of the node in kafka cluster and when i rebooted that node, the problem went away.
问题:
组元数据存储在哪里?问题可能与损坏的动物园管理员有关吗?
Where is group metadata stored ? Can be problem be related to corrupted zookeeper ?
推荐答案
这是作为 bug 在这里提出的 issue.apache.org/jira/browse/KAFKA-9935 with重复 https://issues.apache.org/jira/browse/KAFKA-9752
This was raised as bug here issues.apache.org/jira/browse/KAFKA-9935 with duplicate https://issues.apache.org/jira/browse/KAFKA-9752
自 3 月以来，2.2.3、2.3.2、2.4.2 和 2.5 及更高版本似乎已修复此问题，因此请确保使用最新版本.
This now appears to be fixed since March for versions 2.2.3, 2.3.2, 2.4.2 and 2.5 and above so make sure to use an up to date version.
这篇关于即使没有消费者，消费者群体仍陷于“重新平衡"的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
