卡夫卡消费者滞后于JMX [英] Kafka consumer lag through JMX

问题描述

我正在尝试监控某个消费者群体在卡夫卡0.10中的滞后时间.

I'm trying to monitor the lag of a consumer group in Kafka 0.10.

我们的消费者正在跟踪他们在Kafka(而非ZooKeper)中的偏移量.这意味着我可以使用以下方法获得数字:

Our consumers are keeping track of their offsets in Kafka rather than ZooKeper. This means I can get the figures using the following:

bin/kafka-consumer-groups.sh --bootstrap-server <broker> --describe --group <group-name>

这很好.但是，我的经纪人已经使用 Prometheus JMX导出器来收集许多统计信息.我已将JConsole连接到代理，但无法看到kafka-consumer-groups.sh报告的JMX报告的数据.

This works fine. However, my broker already makes use of the Prometheus JMX exporter for collecting a number of stats. I've connected JConsole to the brokers but can't see the same data being reported in JMX as reported by kafka-consumer-groups.sh.

是否有必要使用JMX 从Kafka那里获取此信息，而无需任何其他工具?

Is there anyway to get this information from Kafka with JMX without needing any additional tools?

推荐答案

您可以为所有分区检索指标kafka.consumer:type=consumer-fetch-manager-metrics,client-id={client-id}的属性{topic}-{partition}.records-lag.那应该等于consumer-groups.sh

You could retrieve the atrributes {topic}-{partition}.records-lag of metric kafka.consumer:type=consumer-fetch-manager-metrics,client-id={client-id} for all partitions. That should be equivalent to the output of consumer-groups.sh

这篇关于卡夫卡消费者滞后于JMX的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！