什么是主题"__consumer_offsets"?在卡夫卡 [英] what is this topic "__consumer_offsets" in Kafka

问题描述

运行此命令时，我得到2个主题.我知道我创建了测试主题，但看到了另一个名为"__consumer_offsets"的主题.顾名思义，它暗示着它与消费者补偿有关，但是它是如何使用的呢?

When I run this command, I get 2 topics. I know I created the test topic but I see an additional topic called "__consumer_offsets". From the name it implies that it is related to consumer offsets, but how is it being used?

$ bin/kafka-topics.sh --list --zookeeper本地主机:2181__consumer_offsets测试

$ bin/kafka-topics.sh --describe --zookeeper localhost:2181
Topic:__consumer_offsets        PartitionCount:50       ReplicationFactor:1     Configs:segment.bytes=104857600,cleanup.policy=compact,compression.type=producer
        Topic: __consumer_offsets       Partition: 0    Leader: 0       Replicas: 0     Isr: 0
        Topic: __consumer_offsets       Partition: 1    Leader: 0       Replicas: 0     Isr: 0
                      *
                      *
                      *
        Topic: __consumer_offsets       Partition: 48    Leader: 0       Replicas: 0     Isr: 0
        Topic: __consumer_offsets       Partition: 49    Leader: 0       Replicas: 0     Isr: 0

这是在Kafka 1.1.0中发生的，为什么会有50个分区.还在寻找一种禁用此方法的方法，因为每次我尝试运行描述"主题时，它都会先打印__consumer_offsets的50个分区，然后再打印我的主题.

This is happening in Kafka 1.1.0 and why there are 50 partitions. Also looking for a way to disable this because every time I try to run "describe" the topics, first it prints the 50 partitions of the __consumer_offsets and then prints my topics.

推荐答案

在Kafka的初始版本中，偏移量是由zookeeper管理的，但随着时间的推移，Kafka不断发展，引入了许多新功能.现在，Kafka在内部/系统级主题(即__consumer_offsets)中管理偏移量.

In initial versions of Kafka, offset was being managed at zookeeper, but Kafka has continuously evolved over the time introducing lot of new features. Now Kafka manages the offset in an internal/system level topic i.e. __consumer_offsets.

无论何时创建主题而未明确指定分区数，默认情况下，Kafka都会为该主题创建50个分区.主题__consumer_offsets同样如此.

Whenever you create a topic without specifying the number of partitions explicitly , Kafka ends up creating 50 partitions by default for that topic. Same implies to the topic __consumer_offsets.

这篇关于什么是主题"__consumer_offsets"?在卡夫卡的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！