Zookeeper 是 Kafka 的必备品吗? [英] Is Zookeeper a must for Kafka?

问题描述

在 Kafka 中，我只想使用单个代理、单个主题和具有一个生产者和多个消费者的单个分区(每个消费者从代理获取自己的数据副本).鉴于此，我不想要使用 Zookeeper 的开销；我不能只使用经纪人吗?为什么必须要有动物园管理员?

In Kafka, I would like to use only a single broker, single topic and a single partition having one producer and multiple consumers (each consumer getting its own copy of data from the broker). Given this, I do not want the overhead of using Zookeeper; Can I not just use the broker only? Why is a Zookeeper must?

推荐答案

是的，运行 Kafka 需要 Zookeeper.来自 Kafka 入门文档:

Yes, Zookeeper is required for running Kafka. From the Kafka Getting Started documentation:

第 2 步:启动服务器

Step 2: Start the server

Kafka 使用zookeeper，所以你需要先启动一个zookeeper 服务器，如果你还没有.您可以使用便利脚本与kafka一起打包，得到一个又快又脏的单节点zookeeper实例.

Kafka uses zookeeper so you need to first start a zookeeper server if you don't already have one. You can use the convenience script packaged with kafka to get a quick-and-dirty single-node zookeeper instance.

至于为什么，人们很久以前就发现您需要某种方式来协调分布式系统中的任务、状态管理、配置等.一些项目已经建立了自己的机制(想想 MongoDB 分片集群中的配置服务器，或者 Elasticsearch 集群中的 Master 节点).其他人选择利用 Zookeeper 作为通用分布式进程协调系统.所以 Kafka、Storm、HBase、SolrCloud 等等都使用 Zookeeper 来帮助管理和协调.

As to why, well people long ago discovered that you need to have some way to coordinating tasks, state management, configuration, etc across a distributed system. Some projects have built their own mechanisms (think of the configuration server in a MongoDB sharded cluster, or a Master node in an Elasticsearch cluster). Others have chosen to take advantage of Zookeeper as a general purpose distributed process coordination system. So Kafka, Storm, HBase, SolrCloud to just name a few all use Zookeeper to help manage and coordinate.

Kafka 是一个分布式系统，专为使用 Zookeeper 而构建.您没有使用 Kafka 的任何分布式功能这一事实不会改变它的构建方式.在任何情况下，使用 Zookeeper 都不应该有太多开销.一个更大的问题是为什么要使用这种特殊的设计模式——Kafka 的单个代理实现错过了多代理集群的所有可靠性特性及其扩展能力.

Kafka is a distributed system and is built to use Zookeeper. The fact that you are not using any of the distributed features of Kafka does not change how it was built. In any event there should not be much overhead from using Zookeeper. A bigger question is why you would use this particular design pattern -- a single broker implementation of Kafka misses out on all of the reliability features of a multi-broker cluster along with its ability to scale.

这篇关于Zookeeper 是 Kafka 的必备品吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！