将协调器标记为组死亡(Kafka) [英] Marking the coordinator dead for group(Kafka)

问题描述

我的一项服务在几个小时前停止工作.我已经重新启动了服务器，但是这个消费者进程没有注册到 Kafka.这是消费者启动时显示的内容:

One of my services stopped working a few hours ago. I have restarted the server but this consumer process just isn't registering with Kafka. This is what shows up when the consumer is started :

2017-04-21 10:22:54.887  INFO 18036 --- [afka-consumer-1] o.a.k.c.c.internals.ConsumerCoordinator  : Revoking previously assigned partitions [] for group mysql-conversions-group
2017-04-21 10:22:54.890  INFO 18036 --- [afka-consumer-1] o.s.k.l.KafkaMessageListenerContainer    : partitions revoked:[]
2017-04-21 10:22:54.890  INFO 18036 --- [afka-consumer-1] o.a.k.c.c.internals.AbstractCoordinator  : (Re-)joining group mysql-conversions-group
2017-04-21 10:22:54.915  INFO 18036 --- [afka-consumer-1] o.a.k.c.c.internals.AbstractCoordinator  : Marking the coordinator li819-138.members.linode.com:9092 (id: 2147483646 rack: null) dead for group mysql-conversions-group

我有 4 个不同的消费者群体，他们运行得很好.有什么想法吗?

I have 4 different consumer groups and they are running just fine. Any thoughts?

更新

真的很奇怪.我们不得不重新启动另一个消费者，重新启动后，一秒钟前工作正常的另一个消费者现在正在记录相同的错误.基本上，我们现在有 2 个消费者进程由于某种原因无法连接到 Kafka.

Its odd really. We had to restart another one of our Consumer, and after restart, the other consumer that was working fine a second ago is now logging the same error. Basically, we now have 2 consumer processes that can't connect to Kafka for some reason.

推荐答案

回答我自己的问题:

终于开始工作了.这不是经纪人问题或消费者问题.发生的事情是，我们在晚上 8:30 左右在我们的两个 Kafka 代理上启用了防火墙.已经与 Kafka 连接的消费者工作正常.但是，当这些消费者之一重新启动时，他们无法连接到协调器节点.我们在两个 Kafka 代理上都禁用了防火墙，数据立即转发给了消费者.这是一个临时修复 ATM，但至少我们有一些东西.

Finally got it to work. This was not a broker issue or a consumer issue. What happened was, we had enabled firewall on both of our Kafka brokers around 8:30 in the night. The consumers that were already connected with Kafka were working fine. However, when one of these consumers were restarted, they were unable to connect to the coordinator node. We disabled firewall on both the Kafka brokers, and the data was immediately relayed to the consumers. It is a temporary fix ATM, but at least we have something.

这篇关于将协调器标记为组死亡(Kafka)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！