这个主题是什么“__consumer_offsets"?在卡夫卡 [英] what is this topic "__consumer_offsets" in Kafka

问题描述

当我运行这个命令时，我得到 2 个主题.我知道我创建了测试主题，但我看到了一个名为__consumer_offsets"的附加主题.顾名思义，它与消费者偏移量有关，但它是如何使用的?

When I run this command, I get 2 topics. I know I created the test topic but I see an additional topic called "__consumer_offsets". From the name it implies that it is related to consumer offsets, but how is it being used?

$ bin/kafka-topics.sh --list --zookeeper localhost:2181__consumer_offsets测试

$ bin/kafka-topics.sh --describe --zookeeper localhost:2181
Topic:__consumer_offsets        PartitionCount:50       ReplicationFactor:1     Configs:segment.bytes=104857600,cleanup.policy=compact,compression.type=producer
        Topic: __consumer_offsets       Partition: 0    Leader: 0       Replicas: 0     Isr: 0
        Topic: __consumer_offsets       Partition: 1    Leader: 0       Replicas: 0     Isr: 0
                      *
                      *
                      *
        Topic: __consumer_offsets       Partition: 48    Leader: 0       Replicas: 0     Isr: 0
        Topic: __consumer_offsets       Partition: 49    Leader: 0       Replicas: 0     Isr: 0

这发生在 Kafka 1.1.0 以及为什么有 50 个分区.也在寻找禁用此功能的方法，因为每次我尝试运行描述"主题时，它首先打印 __consumer_offsets 的 50 个分区，然后打印我的主题.

This is happening in Kafka 1.1.0 and why there are 50 partitions. Also looking for a way to disable this because every time I try to run "describe" the topics, first it prints the 50 partitions of the __consumer_offsets and then prints my topics.

推荐答案

在 Kafka 的初始版本中，offset 由 zookeeper 管理，但随着时间的推移，Kafka 不断发展，引入了许多新功能.现在 Kafka 管理内部/系统级主题中的偏移量，即 __consumer_offsets.

In initial versions of Kafka, offset was being managed at zookeeper, but Kafka has continuously evolved over the time introducing lot of new features. Now Kafka manages the offset in an internal/system level topic i.e. __consumer_offsets.

每当你在没有明确指定分区数的情况下创建一个主题时，Kafka 最终会默认为该主题创建 50 个分区.与 __consumer_offsets 主题相同.

Whenever you create a topic without specifying the number of partitions explicitly , Kafka ends up creating 50 partitions by default for that topic. Same implies to the topic __consumer_offsets.

这篇关于这个主题是什么“__consumer_offsets"?在卡夫卡的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！