为什么 Kafka 会警告“分区具有没有匹配侦听器的领导代理"? [英] Why could Kafka warn "partitions have leader brokers without a matching listener"?

问题描述

我试图让 Kafka 第一次在 docker-compose 上工作.该应用程序在没有 docker 的情况下运行良好.但是在 docker 上，我收到了如下所述的错误.Kafka 会抛出此错误的任何原因?

错误:

<块引用>

电子邮件服务_1 |2018-12-01 14:32:02.448 警告 1 ---[ntainer#0-0-C-1] o.a.k.c.NetworkClient : [消费者clientId=consumer-2, groupId=kafka] 1 个分区有 leader brokers没有匹配的侦听器，包括 [email-token-0]

我的 docker-compose 配置:

版本:'3.3'服务:动物园管理员:图片:wurstmeister/动物园管理员端口:- 2181:2181"卡夫卡:图片:wurstmeister/kafka命令:[start-kafka.sh]环境:KAFKA_ZOOKEEPER_CONNECT:动物园管理员:2181KAFKA_ADVERTISED_HOST_NAME:192.168.23.134KAFKA_CREATE_TOPICS: "email-token:1:1"卷:-/var/run/docker.sock:/var/run/docker.sock端口:- 9092:9092"取决于:- 动物园管理员电子邮件服务:构建:./email-service环境:SPRING_KAFKA_BOOTSTRAPSERVERS:卡夫卡:9092端口:- 8081:8081"取决于:- 卡夫卡

解决方案

如对您的问题的评论所述，问题似乎与 Kafka 代理的广告名称有关.根据您的 docker-compose 您应该使用 192.168.23.134 但您的电子邮件服务正在使用 kafka:9092.您可以尝试使用此 docker-compose.我用 confluentinc 提供的最新 Zookeeper 和 Kafka 替换了 wurstmeister 服务，并添加了您的电子邮件服务.

---版本:'2'服务:动物园管理员:图片:confluentinc/cp-zookeeper:最新环境:ZOOKEEPER_CLIENT_PORT:2181ZOOKEEPER_TICK_TIME:2000卡夫卡:图片:confluentinc/cp-kafka:最新取决于:- 动物园管理员端口:- 9092:9092环境:KAFKA_BROKER_ID:1KAFKA_ZOOKEEPER_CONNECT:动物园管理员:2181KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:29092,PLAINTEXT_HOST://localhost:9092KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXTKAFKA_INTER_BROKER_LISTENER_NAME:PLAINTEXTKAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR:1电子邮件服务:构建:./email-service环境:SPRING_KAFKA_BOOTSTRAPSERVERS:卡夫卡:29092端口:- 8081:8081"取决于:- 卡夫卡

<块引用>

advertised.listeners:发布到 ZooKeeper 供客户端使用的侦听器，如果与侦听器配置属性不同.在 IaaS 环境中，这可能需要与代理绑定的接口不同.如果未设置，则将使用侦听器的值.与侦听器不同，通告 0.0.0.0 元地址是无效的.

请注意，KAFKA_ADVERTISED_HOST_NAME 已被弃用，建议改用 KAFKA_ADVERTISED_LISTENERS.有关KAFKA_ADVERTISED_LISTENERS 的更多信息，请查看此处.

I'm trying to get Kafka to work on docker-compose for the first time. The application runs fine without docker. But on docker, I get the error as described below. Any reason why Kafka would throw this error?

The error:

email-service_1 | 2018-12-01 14:32:02.448 WARN 1 --- [ntainer#0-0-C-1] o.a.k.c.NetworkClient : [Consumer clientId=consumer-2, groupId=kafka] 1 partitions have leader brokers without a matching listener, including [email-token-0]

My docker-compose config:

version: '3.3'
    services:
     zookeeper:
      image: wurstmeister/zookeeper
      ports:
       - "2181:2181"

 kafka:
  image: wurstmeister/kafka
  command: [start-kafka.sh]
  environment:
   KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
   KAFKA_ADVERTISED_HOST_NAME: 192.168.23.134
   KAFKA_CREATE_TOPICS: "email-token:1:1"
  volumes:
   - /var/run/docker.sock:/var/run/docker.sock
  ports:
   - "9092:9092"
  depends_on:
   - zookeeper

 email-service:
  build: ./email-service
  environment:
   SPRING_KAFKA_BOOTSTRAPSERVERS: kafka:9092
  ports:
   - "8081:8081"
  depends_on:
   - kafka

解决方案

As stated in the comments to your question the problem seems to be with the advertised name for the Kafka broker. According to your docker-compose you should be using 192.168.23.134 but your email-service is using kafka:9092. You can try with this docker-compose. I replaced the wurstmeister services with the latest Zookeeper and Kafka provided by confluentinc and added your email-service.

---
version: '2'
services:
zookeeper:
    image: confluentinc/cp-zookeeper:latest
    environment:
    ZOOKEEPER_CLIENT_PORT: 2181
    ZOOKEEPER_TICK_TIME: 2000

kafka:
    image: confluentinc/cp-kafka:latest
    depends_on:
    - zookeeper
    ports:
    - 9092:9092
    environment:
    KAFKA_BROKER_ID: 1
    KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
    KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:29092,PLAINTEXT_HOST://localhost:9092
    KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
    KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
    KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1

email-service:
  build: ./email-service
  environment:
   SPRING_KAFKA_BOOTSTRAPSERVERS: kafka:29092
  ports:
   - "8081:8081"
  depends_on:
   - kafka

advertised.listeners: Listeners to publish to ZooKeeper for clients to use, if different than the listeners config property. In IaaS environments, this may need to be different from the interface to which the broker binds. If this is not set, the value for listeners will be used. Unlike listeners it is not valid to advertise the 0.0.0.0 meta-address.

Please note that KAFKA_ADVERTISED_HOST_NAME has been deprecated and it's recommended to use KAFKA_ADVERTISED_LISTENERS instead. For more information about KAFKA_ADVERTISED_LISTENERS check here.

这篇关于为什么 Kafka 会警告“分区具有没有匹配侦听器的领导代理"?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！