如何检查主题是否被Kafka中的消费者消费 [英] How to check if a topic was consumed by a consumer in Kafka

问题描述

应该如何检查特定的消费群体是否完全消费了一个主题?
这相当于在标准排队系统中检查队列是否为空.
这并不明显，因为每个消费者只能看到给定的分区，因此不知道该组中的其他消费者是否消费了他们的部分.
我想这应该由 AdminUtils 或 ZkUtils 以某种方式完成，但我似乎找不到正确的方法.

How should one check if a specific consumer group, fully consumed a topic?
This is the equivalent of checking if a queue is empty in standard queuing system.
This isn't obvious as each consumer only see the partitions it is given hence it isn't aware if other consumers in it's group consumed their part.
I guess this should be done somehow by AdminUtils or ZkUtils but I cant seem to find the right way.

推荐答案

请使用kafka-consumer-groups.sh查看消费者偏移量.
(kafka-consumer-offset-checker.sh 具有类似的功能，但在 0.9.0.0 中已被弃用.)

Please use kafka-consumer-groups.sh to check consumer offset.
(The kafka-consumer-offset-checker.sh has similar functionality but has been deprecated in 0.9.0.0.)

bin/kafka-consumer-groups.sh --bootstrap-server xxx:9092--groupxxx-group-id --describe --new-consumer
小组话题分区CURRENT-OFFSET LOG-END-OFFSET LAG OWNER consumer.test.0529.095217Secured_topic 0 586 3333 2747 消费者-1_/10.139.0.4消费者.test.0529.095217secure_topic 3334 3334 0消费者1_/10.139.0.4消费者.test.0529.095217secure_topic 2 03333 3333 消费者-1_/10.139.0.4

bin/kafka-consumer-groups.sh --bootstrap-server xxx:9092--group xxx-group-id --describe --new-consumer
GROUP TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG OWNER consumer.test.0529.095217 secured_topic 0 586 3333 2747 consumer-1_/10.139.0.4 consumer.test.0529.095217 secured_topic 3334 3334 0 consumer-1_/10.139.0.4 consumer.test.0529.095217 secured_topic 2 0 3333 3333 consumer-1_/10.139.0.4

这篇关于如何检查主题是否被Kafka中的消费者消费的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！