Kafka访问docker内部和外部 [英] Kafka access inside and outside docker

问题描述

我正在尝试使用docker-compose启动kafka服务，并且应该可以在docker内部和外部对其进行访问。因此，应该在内部和外部设置合适的广告客户：

 版本：'3'
服务： 
动物园管理员：
图片：wurstmeister / zookeeper 
端口：
- 2181：2181 
 kafka：
图片：wurstmeister / kafka 
端口：
- 9094：9092 
环境：
 KAFKA_ZOOKEEPER_CONNECT：zookeeper：2181 
 KAFKA_LISTENER_SECURITY_PROTOCOL_MAP：INSIDE：PLAINTEXT，OUTSIDE：PLAINTEXT 
 KAFKA_LISTENERS：INSIDE：// ：9092，OUTSIDE：//127.0.0.1：9094 
 KAFKA_ADVERTISED_LISTENERS：INSIDE：//：9092，OUTSIDE：//127.0.0.1：9094 
 KAFKA_INTER_BROKER_LISTENER_NAME：INSIDE 
  

问题是，当我尝试从集群外部连接时，我没有得到127.0.0.1作为节点名称，但是内部主机名：

  $ kafkacat -L -b 127.0.0.1:9094 
所有主题的元数据（来自代理-1：127.0.0.1:9092/bootstrap）：
 1个经纪人： 
经纪人1001，地址为91588ea968d4：9092 
 28个主题：
 ... 
  

KAFKA_ADVERTISED_LISTENERS和KAFKA_LISTENERS的目的不是处理这种情况吗？我尝试设置KAFKA_ADVERTISED_HOST_NAME，但是它被忽略了（一个文档说它已被弃用，另一个说它仍然处于活动状态），但这似乎不是答案，因为我想要为两个不同的网络使用两个不同的公告主机名。 / p>

我想仍然存在一个老问题：如何使kafka在docker-compose内部和外部工作？

解决方案

您的侦听器配置看起来正确，问题出在您的Docker Compose：

 端口： 
- 9094：9092 
  

您正在映射 9094 （外部）回到 9092 （内部），因此当您连接时，您将连接到内部侦听器。如果删除此配置行，则您的侦听器设置应该可以正常工作。

有关更多信息，请参见 http://rmoff.net/2018/08/02/kafka-listeners-explained/

I'm trying to start a kafka service using docker-compose, and it should be able to be accessed inside and outside docker. So, it should be matter of setting the right advertisers inside and outside:

version: '3'
services:
  zookeeper:
    image: wurstmeister/zookeeper
    ports:
      - "2181:2181"
  kafka:
    image: wurstmeister/kafka
    ports:
      - "9094:9092"
    environment:
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INSIDE:PLAINTEXT,OUTSIDE:PLAINTEXT
      KAFKA_LISTENERS: INSIDE://:9092,OUTSIDE://127.0.0.1:9094
      KAFKA_ADVERTISED_LISTENERS: INSIDE://:9092,OUTSIDE://127.0.0.1:9094
      KAFKA_INTER_BROKER_LISTENER_NAME: INSIDE

The problem is that when I try to connect from outside the cluster, I don't get 127.0.0.1 as the name of the node, but the internal hostname:

$ kafkacat -L -b 127.0.0.1:9094
Metadata for all topics (from broker -1: 127.0.0.1:9092/bootstrap):
 1 brokers:
  broker 1001 at 91588ea968d4:9092
 28 topics:
 ...

Isn't the purpose of KAFKA_ADVERTISED_LISTENERS and KAFKA_LISTENERS to handle that situation? I tried setting KAFKA_ADVERTISED_HOST_NAME but it's ignored (one piece of documentation says it's deprecated, other one says that it's still active), but nevertheless that doesn't seem to be the answer, since I want two different advertised hostnames for two different networks.

I guess the old question remains: how to make kafka work inside and outside docker-compose?

解决方案

Your config of the listeners looks correct, the problem is with your Docker Compose:

ports:
  - "9094:9092"

You’re mapping 9094 (‘Outside’) back to 9092 (‘Inside’) and thus when you connect you’re connecting to the ‘Inside’ listener. If you remove this line of config then your listener set up should work as intended.

For more info, see http://rmoff.net/2018/08/02/kafka-listeners-explained/

这篇关于Kafka访问docker内部和外部的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！