zookeeper 在哪里存放 kafka 集群和相关信息? [英] Where does zookeeper store kafka cluster and related information?
问题描述
通过说集群信息，我指的是像
By saying cluster info, I am referring to information like
- 订阅的消费者/消费者群体
- 读取和提交的偏移量
- 分区的领导者和追随者
- 服务器上的主题等
zookeeper 是将这些信息保存在它自己的数据库中(尽管我从来没有听说过 Zookeeper 有任何自己的数据库)还是将这些信息存储在 Kafka 集群中的某些主题等?
Does zookeeper keep this info in its own db (though I never heard of zookeeper having any db of its own till date) or it stores this info in the Kafka cluster on some topics etc?
以及后续问题:Zookeeper 如何从 __consumer_offsets 主题检索消费者偏移量?
推荐答案
ZooKeeper 数据模型ZooKeeper 有一个分层的命名空间，很像一个分布式文件系统.唯一的区别是命名空间中的每个节点都可以拥有与其关联的数据以及子节点.这就像拥有一个允许文件也成为目录的文件系统.节点的路径总是表示为规范的、绝对的、斜线分隔的路径；没有相对参考.
The ZooKeeper Data Model ZooKeeper has a hierarchal name space, much like a distributed file system. The only difference is that each node in the namespace can have data associated with it as well as children. It is like having a file system that allows a file to also be a directory. Paths to nodes are always expressed as canonical, absolute, slash-separated paths; there are no relative reference.
所以没有数据库.
偏移量保存在kafka中，而不是zookeeper中，在这个主题中:__consumer_offsets
Offsets are kept in kafka, not in zookeeper, in this topic: __consumer_offsets
在zookeeper节点中:
In a zookeeper node:
ls/brokers/ids
ls /brokers/ids
ls/brokers/topics
ls /brokers/topics
获取/brokers/ids/0
get /brokers/ids/0
在这里你可以阅读更多关于zookeeper中的kafka数据结构:Kafka 数据结构zookeeper
here you can read more about the kafka datastructures in zookeeper: Kafka datastructures zookeeper
有一个 KIP 可以完全删除 zookeeper:KIP 移除zookeeper
There is a KIP for removing zookeeper alltogether: KIP Remove zookeeper
这篇关于zookeeper 在哪里存放 kafka 集群和相关信息?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
