docker内部和外部的Kafka访问 [英] Kafka access inside and outside docker

问题描述

我正在尝试使用 docker-compose 启动一个 kafka 服务，它应该可以在 docker 内部和外部访问.因此，应该在内部和外部设置合适的广告客户:

I'm trying to start a kafka service using docker-compose, and it should be able to be accessed inside and outside docker. So, it should be matter of setting the right advertisers inside and outside:

version: '3'
services:
  zookeeper:
    image: wurstmeister/zookeeper
    ports:
      - "2181:2181"
  kafka:
    image: wurstmeister/kafka
    ports:
      - "9094:9092"
    environment:
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INSIDE:PLAINTEXT,OUTSIDE:PLAINTEXT
      KAFKA_LISTENERS: INSIDE://:9092,OUTSIDE://127.0.0.1:9094
      KAFKA_ADVERTISED_LISTENERS: INSIDE://:9092,OUTSIDE://127.0.0.1:9094
      KAFKA_INTER_BROKER_LISTENER_NAME: INSIDE

问题是，当我尝试从集群外部连接时，我得到的不是 127.0.0.1 作为节点名称，而是内部主机名:

The problem is that when I try to connect from outside the cluster, I don't get 127.0.0.1 as the name of the node, but the internal hostname:

$ kafkacat -L -b 127.0.0.1:9094
Metadata for all topics (from broker -1: 127.0.0.1:9092/bootstrap):
 1 brokers:
  broker 1001 at 91588ea968d4:9092
 28 topics:
 ...

KAFKA_ADVERTISED_LISTENERS 和 KAFKA_LISTENERS 的目的不是处理这种情况吗?我尝试设置 KAFKA_ADVERTISED_HOST_NAME 但它被忽略了(一份文件说它已弃用，另一份说它仍然处于活动状态)，但这似乎不是答案，因为我想要两个不同的广告主机名用于两个不同的网络.

Isn't the purpose of KAFKA_ADVERTISED_LISTENERS and KAFKA_LISTENERS to handle that situation? I tried setting KAFKA_ADVERTISED_HOST_NAME but it's ignored (one piece of documentation says it's deprecated, other one says that it's still active), but nevertheless that doesn't seem to be the answer, since I want two different advertised hostnames for two different networks.

我想老问题仍然存在:如何让 kafka 在 docker-compose 内外工作?

I guess the old question remains: how to make kafka work inside and outside docker-compose?

推荐答案

您的侦听器配置看起来正确，问题在于您的 Docker Compose:

Your config of the listeners looks correct, the problem is with your Docker Compose:

ports:
  - "9094:9092"

您正在将 9094 ('Outside') 映射回 9092 ('Inside')，因此当您连接时，您正在连接到 'Inside' 侦听器.如果您删除这行配置，那么您的侦听器设置应该按预期工作.

You’re mapping 9094 (‘Outside’) back to 9092 (‘Inside’) and thus when you connect you’re connecting to the ‘Inside’ listener. If you remove this line of config then your listener set up should work as intended.

有关详细信息，请参阅 http://rmoff.net/2018/08/02/kafka-listeners-解释/

For more info, see http://rmoff.net/2018/08/02/kafka-listeners-explained/

这篇关于docker内部和外部的Kafka访问的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！