单独的zookeeper安装与否使用kafka 10.2? [英] Separate zookeeper install or not using kafka 10.2?
问题描述
我想使用Kafka 10.2自带的内嵌Zookeeper 3.4.9，不单独安装Zookeeper.每个 Kafka 代理在本地主机上总是有一个 1:1 的 Zookeeper.
I would like to use the embedded Zookeeper 3.4.9 that come with Kafka 10.2, and not install Zookeeper separately. Each Kafka broker will always have a 1:1 Zookeeper on localhost.
因此，如果我在主机 A、b、C、D 和 E 上有 5 个代理，每个代理都运行一个 Kafka 和 Zookeeper 实例，那么仅运行 Kafka 提供的 Zookeeper 就足够了吗?
So if I have 5 brokers on hosts A, b, C, D and E, each with a single Kafka and Zookeeper instance running on them, then is it sufficient to just run the Zookeeper provided with Kafka?
与独立版本相比，嵌入式 3.4.9 Zookeper 有哪些缺点或配置限制?
What downsides or configuration limitations if any does the embedded 3.4.9 Zookeper have compared to the standalone version?
推荐答案
这些是不在与 Kafka brokers 相同的机器上运行 zookeeper 的几个原因.
These are a few reason not to run zookeeper on the same box as Kafka brokers.
它们的比例不同
They scale differently
5 zk 和 5 Kafka 有效，但 6:6 或 11:11 无效.即使对于相当大的 Kafka 集群，您也不需要超过 5 个 zookeeper 节点.与 Kafka 不同，Zookeeper 将数据复制到所有节点，因此随着您添加更多节点，它会变慢.
5 zk and 5 Kafka works but 6:6 or 11:11 do not. You don't need more than 5 zookeeper nodes even for a quite large Kafka cluster. Unlike Kafka, Zookeeper replicates data to all nodes so it gets slower as you add more nodes.
它们争夺磁盘 I/O
Zookeeper 对磁盘 I/O 延迟非常敏感.您需要将它放在与 Kafka 提交日志不同的物理磁盘上，否则您可能会面临大量发布到 Kafka 会减慢 zookeeper 速度并导致它退出集合的风险，从而导致潜在的问题.
Zookeeper is very disk I/O latency sensitive. You need to have it on a separate physical disk from the Kafka commit log or you run the risk that a lot of publishing to Kafka will slow zookeeper down and cause it to drop out of the ensemble causing potential problems.
它们争夺页面缓存
Kafka 使用 Linux OS 页面缓存来减少磁盘 I/O.当其他应用程序与 Kafka 在同一台机器上运行时，您会使用从 Kafka 缓存中取出的其他数据减少或污染"页面缓存.
Kafka uses Linux OS page cache to reduce disk I/O. When other apps run on the same box as Kafka you reduce or "pollute" the page cache with other data that takes away from cache for Kafka.
服务器故障导致更多基础设施瘫痪
Server failures take down more infrastructure
如果盒子重启，你会同时失去一个动物园管理员和一个经纪人.
If the box reboots you lose both a zookeeper and a broker at the same time.
这篇关于单独的zookeeper安装与否使用kafka 10.2?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
