Kafka分区和偏移量消失了 [英] Kafka partitions and offsets disappeared

问题描述

我的Kafka客户端在启用了自动缩放的GCP App Engine Flex环境中运行(由于CPU使用率较低，GCP会将实例计数至少保留为2，并且大多数情况下为2).在2个虚拟机上运行的使用者组使用了来自20个分区中各种主题的消息已有几个月了，最近我注意到，较旧主题中的分区缩小为1(！)，该使用者组的偏移量重置为0.[partition]目录也从kafka-logs目录中消失了.奇怪的是，最近创建的主题分区是完整的.我有3个不同的环境(都在GCP中)，这一切都发生在这三个环境中.我们没有看到任何丢失的消息或数据问题，但想了解发生了什么，避免再次发生这种情况.

My Kafka clients are running in GCP App Engine Flex environment with auto scale enabled (GCP keeps the instance count to at least two and it has been mostly 2 due to low CPU usages). The consumer groups running in that 2 VMs have been consuming messages from various topics in 20 partitions for several months and recently I noticed that partitions in older topics shrank to just 1 (!) and offsets for that consumer group was reset to 0. topic-[partition] directories were also gone from the kafka-logs directory. Strangely, recently created topic partitions are intact. I have 3 different environments (all in GCP) and this happened to all three. We didn't see any lost messages or data problem but want to understand what had happened to avoid this happening again.

kafka代理和zookeeper在同一GCP计算引擎实例中运行(我知道这不是最佳实践，并且有计划进行改进)，我怀疑这与机器重启有关，并且抹去了一些信息.但是，我验证了数据文件写在/opt/bitnami/(kafka | bitnami)目录下，而不是/tmp目录下，可以通过计算机重新启动将其删除.

The kafka broker and zookeeper are running in the same and single GCP compute engine instance (I know it's not the best practice and have plan to improve) and I suspect it has something to do with machine restart and that wipes out some information. However, I verified that data files are written under /opt/bitnami/(kafka|bitnami) directory and not /tmp which can be removed by machine restarts.

• 春季卡夫卡1.1.3
• kafka客户端0.10.1.1
• 单节点kafka经纪人0.10.1.0
• 单节点zookeeper 3.4.9

任何对此的见解将不胜感激！

Any insights on this will be appreciated!

推荐答案

此处是Bitnami开发人员.我可以重现该问题并将其跟踪到一个初始化脚本，该脚本清除了 tmp/kafka-logs/文件夹的内容.

Bitnami developer here. I could reproduce the issue and track it down to an init script that was clearing the content of the tmp/kafka-logs/ folder.

我们发布了kafka 安装程序，

We released a new revision of the kafka installers, virtual machines and cloud images fixing the issue. The revision that includes the fix is 1.0.0-2.

这篇关于Kafka分区和偏移量消失了的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！