Kafka群集部署配置的最佳做法? [英] Best Practices for Kafka Cluster Deployment Configuration?

问题描述

我在这里要求一般最佳实践:

I'm asking for general best practices here:

如果我要一个五节点集群，是否所有五个节点都运行Confluent Platform Umbrella Packages，其中包括Zookeeper，Kafka，schema-registry?

If I want a five node cluster, do all five nodes run the Confluent Platform Umbrella Packages that include Zookeeper, Kafka, schema-registry?

是否曾经建议在与Kafka群集不同的服务器上运行zookeper群集?

Is it ever recommended to run the zookeper cluster on separate servers from the Kafka cluster?

如果我要运行Kafka Connect分布式工作程序，是否可以在所有群集节点上运行它?我是否要在单独的服务器上运行?为此推荐使用Docker还是不需要Docker?

If I want to run the Kafka Connect distributed worker, do I run that on all cluster nodes? Do I ever want to run on separate servers? Is Docker recommended for this or is Docker unnecessary?

使用Kafka Streaming应用程序时，它们是否应在所有群集节点上运行?他们应该被码头化吗?它们应该在单独的节点上运行吗?

With Kafka Streaming apps, should they be run on all cluster nodes? Should they be dockerized? Should they ever run on separate nodes?

是否推荐类似Mesos的东西?

Is something like Mesos recommended?

推荐答案

最佳做法是在专用服务器(或虚拟服务器)上运行Kafka Brokers.Zookeeper也是如此.

It is a best practice to run Kafka Brokers on dedicated servers (or virtual servers). The same is true of Zookeeper.

Confluent平台的所有其他组件可以在共同的服务器上或在单独的计算机上共同运行.

All the other components of the Confluent Platform can run colocated on common servers or on separate machines.

通常只运行一个Schema Registry(如果需要容错，则运行两个).它们可以在可以连接回Kafka Brokers的任何计算机上运行.

You would typically run only one Schema Registry (or two if you want fault tolerance). They can run on any machine that can connect back to the Kafka Brokers.

Kafka Connect分布式工作程序仅需要在要托管Kafka连接器的计算机上运行.他们只需要能够重新连接到Kafka经纪人.

Kafka Connect distributed workers only need to run on machines that you want to host Kafka Connectors. They just need to be able to connect back to the Kafka Brokers.

Kafka Streams应用程序可以在您想要的任何地方运行，只要它们可以重新连接到Kafka Brokers.

Kafka Streams apps can run anywhere you want so long as they can connect back to the Kafka Brokers.

所有组件都可以在docker容器中运行，也可以不使用docker.

All components can run inside docker containers or without docker.

您可以使用所需的任何微服务或数据中心资源管理工具(或根本不使用)-这是您的选择.

You can use whatever microservices or data center resource management tools you want (or none at all) - it is your choice.

这篇关于Kafka群集部署配置的最佳做法?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！