Kafka服务器配置-监听程序与播发监听程序 [英] Kafka server configuration - listeners vs. advertised.listeners

问题描述

要使Kafka运行，您需要在 config / server.properties 文件中设置一些属性。我不了解两个设置。

To get Kafka running, you need to set some properties in config/server.properties file. There are two settings I don't understand.

有人可以解释一下监听者和advertised.listeners属性之间的区别吗？

Can somebody explain the difference between listeners and advertised.listeners property?

文档说：

监听器：套接字服务器监听的地址。

listeners: The address the socket server listens on.

和

advertiseds。 ：
代理将向生产者和消费者发布广告的主机名和端口。

advertised.listeners: Hostname and port the broker will advertise to producers and consumers.

何时必须使用哪种设置？

When do I have to use which setting?

推荐答案

由于我无法发表评论，因此我将其发布为答案，并添加到M.Situations答案中。

Since I cannot comment yet I will post this as an "answer", adding on to M.Situations answer.

在他链接的同一个文档中，有一个关于KAFKA客户端使用哪个侦听器的摘要（ https://cwiki.apache.org/confluence/display/KAFKA/KIP-103% 3A +内部+和外部+流量的分离+ ）：

Within the same document he links there is this blurb about which listener is used by a KAFKA client (https://cwiki.apache.org/confluence/display/KAFKA/KIP-103%3A+Separation+of+Internal+and+External+traffic):

如前所述，客户端从不会看到侦听器名称并将完全像以前那样发出元数据请求。区别在于，它们返回的端点列表仅限于发出请求的端点的侦听器名称。

As stated previously, clients never see listener names and will make metadata requests exactly as before. The difference is that the list of endpoints they get back is restricted to the listener name of the endpoint where they made the request.

这很重要，因为取决于您在bootstrap.servers配置中使用的URL，它将是客户端在映射到advertised.listeners中时将获得的URL *（不知道侦听器不存在的行为） 。

This is important as depending on what URL you use in your bootstrap.servers config that will be the URL* that the client will get back if it is mapped in advertised.listeners (do not know what the behavior is if the listener does not exist).

也请注意：

例外是基于ZooKeeper的消费者。这些消费者直接从ZooKeeper检索经纪人注册信息，并将选择第一个使用PLAINTEXT作为安全协议（他们支持的唯一安全协议）的侦听器。

The exception is ZooKeeper-based consumers. These consumers retrieve the broker registration information directly from ZooKeeper and will choose the first listener with PLAINTEXT as the security protocol (the only security protocol they support).

作为示例代理配置（针对集群中的所有代理）：

As an example broker config (for all brokers in cluster):

advertised.listeners = EXTERNAL：//XXXXX.compute -1.amazonaws.com:9990,INTERNAL://ip-XXXXX.ec2.internal:9993

advertised.listeners=EXTERNAL://XXXXX.compute-1.amazonaws.com:9990,INTERNAL://ip-XXXXX.ec2.internal:9993

inter.broker.listener.name = INTERNAL

inter.broker.listener.name=INTERNAL

listener.security.protocol.map = EXTERNAL：SSL，INTERNAL：PLAINTEXT

listener.security.protocol.map=EXTERNAL:SSL,INTERNAL:PLAINTEXT

如果如果客户端使用XXXXX.compute-1.amazonaws.com:9990进行连接，则元数据获取将转到该代理。但是，与组协调器或组长一起使用的返回URL可能是123.compute-1.amazonaws.com:9990*（另一台计算机！）。这意味着无论实际URL（节点）如何，匹配都是按KIP-103所通告的侦听器名称进行的。

If the client uses XXXXX.compute-1.amazonaws.com:9990 to connect, the metadata fetch will go to that broker. However, the returning URL to use with the Group Coordinator or Leader could be 123.compute-1.amazonaws.com:9990* (a different machine!). This means that the match is done on the listener name as advertised by KIP-103 irrespective of the actual URL (node).

由于EXTERNAL的协议映射为SSL，因此会强制您使用SSL密钥库进行连接。

Since the protocol map for EXTERNAL is SSL this would force you to use an SSL keystore to connect.

如果另一方面，您位于AWS内，则可以发出ip-XXXXX.ec2.internal：

If on the other hand you are within AWS lets say, you can then issue ip-XXXXX.ec2.internal:9993 and the corresponding connection would be plaintext as per the protocol map.

这在IaaS中尤其需要，因为在我的情况下，经纪人和消费者生活在AWS上，而我的生产者生活在IaaS中在客户端站点上，因此需要不同的安全协议和侦听器。

This is especially needed in IaaS where in my case brokers and consumers live on AWS, whereas my producer lives on a client site, thus needing different security protocols and listeners.

编辑：
另外，由于您为不同的客户端使用了不同的端口，因此添加入站规则要容易得多。 （经纪人，生产者，消费者）。

Also adding Inbound Rules is much easier now that you have different ports for different clients (brokers, producers, consumers).

EDIT2：
如果以上内容仍不清楚，则本文是深入的指南： https://rmoff.net/2018/0 8/02 / kafka-listeners-explained /

这篇关于Kafka服务器配置-监听程序与播发监听程序的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！