分别说明了Kafka控制台生产者和使用者中的--broker-list选项和--bootstrap-server选项 [英] Clarification on the --broker-list option and --bootstrap-server option in Kafka-console producer and consumer respectively

问题描述

在Kafka-console-producer中，--broker-list包含服务器列表.生产者是否连接到所有人?(或者)生产者是否使用服务器列表连接其中一台服务器，如果该服务器出现故障，则切换到下一个，依此类推?

In Kafka-console-producer, the --broker-list takes a list of servers. Does the producer connect to all of them? (or) Does the producer uses the list of servers to connect one of them and if that one fails, switches to the next and so on?

类似地，在Kafka-console-consumer中，--bootstrap-server列出了Kafka服务器的列表.如果有两个Kafka服务器，是否需要在--bootstrap-server中同时指定两个服务器?我尝试用一​​台服务器(Kafka-server1)运行使用者，当我停止Kafka-server1时，它继续接收该主题的数据.

Similarly, in Kafka-console-consumer the --bootstrap-server takes a list of Kafka servers. If there are two Kafka servers, do I need to specify both of them in the --bootstrap-server? I tried myself running the consumer with one server (Kafka-server1) and when I stopped Kafka-server1, it continued to receive data for the topic.

推荐答案

它们的作用相同.

如果您查看Kafka源代码，则会看到两个选项都导致相同的"bootstrap.servers" 配置属性

If you look at the Kafka source code, you'll see both options lead to the same "bootstrap.servers" configuration property

  def producerProps(config: ProducerConfig): Properties = {
    val props =
      if (config.options.has(config.producerConfigOpt))
        Utils.loadProps(config.options.valueOf(config.producerConfigOpt))
      else new Properties

    props ++= config.extraProducerProps

    props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, config.brokerList) // <---- brokerList is passed as BOOTSTRAP_SERVER

消费者和生产者都将以循环方式连接到提供的地址列表，以创建到Kafka控制器的初始布尔"连接，该控制器知道给定时间集群中的所有可用代理.优良作法是至少提供3个以确保高可用性.

Both consumer and producer will connect in a round-robin fashion to the list of provided addresses to create an initial "boostrap" connection to the Kafka Controller, which knows about all available brokers in the cluster at a given time. It is good practice to give at least 3 for high-availability.

如果有两个Kafka服务器，我是否需要在--bootstrap-server中指定两个服务器?

If there are two Kafka servers, do I need to specify both of them in the --bootstrap-server?

关于可使用多个地址，在云环境中，您可能在可用性区域中拥有代理，建议每个可用性区域至少列出2个代理，因此3个区域中总共有6个代理.

With regards to having multiple addresses availble to use, in a cloud enviornment, where you might have brokers over availability zones, it is recommended to have at least 2 brokers listed per availability zone, so 6 total for 3 zones.

为客户提供的地址可以使用负载均衡器/反向代理简化为单个 kafka.your.network:9092 地址，但是随后您要引入额外的DNS和网络跃点来计算为了拥有一个众所周知的地址而取消了连接.

The address provided for clients could be similified using a load balancer / reverse proxy down to a single kafka.your.network:9092 address, but then you are introducing extra DNS and network hops to figure out the connection for the sake of having a single, well-known, address.

在任何情况下，代理的所有可用地址都将交给客户端，然后在本地缓存.

In any case, all available addresses for the brokers will be handed to the clients and then cached locally.

但是，重要的是要认识到所有发送/轮询请求将仅与TopicPartition的单数领导者进行通信，尽管您给出了多少个地址以及一个主题将具有多少个副本.

However, it is important to recognize all send/poll requests will only communicate with the singular leader of a TopicPartition, despite how many addresses you give and how many replicas a topic will have.

这篇关于分别说明了Kafka控制台生产者和使用者中的--broker-list选项和--bootstrap-server选项的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！