Kafka 内外部通信 [英] Internal and External communication in Kafka

问题描述

流程:本地 ------> 代理 ---> Kafka

Flow: On-Prem ------>Proxy--->Kafka

advertised.listeners=PLAINTEXT://proxyhostname:8080 - for external communication
listeners=PLAINTEXT://:9092 = for internal communication   

•当我们设置两个属性时，内部通信没有发生.(复制问题，消费者无法通过本地连接，我们必须提供消费者通信的代理IP)

•When we set both the property, the internal communication is not happening.(Replication issue and the consumer couldn’t connect via locally and we have to provide the proxy Ip for consumer communication)

•我们如何才能有效地将财产用于内部和外部沟通?

•How we can effectively use both the property for internal and external communication?

•进行外部和内部沟通的任何替代想法?

•Any alternative idea to do the external and internal communication?

推荐答案

定义多个监听器是很常见的，而 Kafka 很好地支持这一点.

It's very common to define multiple listeners and Kafka supports that very well.

要定义多个监听器，您需要在 advertised.listeners/listeners 中列出所有监听器.

To define several listeners, you need to list all of them in advertised.listeners/listeners.

如果多个侦听器要使用相同的安全协议 (PLAINTEXT)，您还需要设置 listener.security.protocol.map 以将自定义名称映射到 Security协议.请参阅 Kafka 文档中的 broker 配置.

If multiple listeners are going to use the same Security Protocol (PLAINTEXT), you also need to set listener.security.protocol.map to map custom names to Security Protocols. See broker configs in the Kafka Docs.

例如:

listener.security.protocol.map=INTERNAL:PLAINTEXT,EXTERNAL:PLAINTEXT
advertised.listeners=INTERNAL://:9092,EXTERNAL://proxyhostname:8080
listeners=INTERNAL://:9092,EXTERNAL://:8080

这将 2 个名称 EXTERNAL 和 INTERNAL(您可以使用您喜欢的任何名称，我只是重用了您问题中的名称)到 PLAINTEXT 安全协议.然后对于每个，它定义要侦听的端口和要在元数据响应中通告的主机名.

This maps 2 names EXTERNAL and INTERNAL (you can use any name you like, I just reused names from your question) to the PLAINTEXT security protocol. Then for each, it defines the port to listen to and the hostname to advertise in metadata responses.

这篇关于Kafka 内外部通信的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！