无法从容器外运行的 Spring Boot 应用程序连接到容器中运行的 Kafka [英] Unable to connect to Kafka run in container from Spring Boot app run outside container

问题描述

我通过以下方式在本地运行 kafka:

I'm running kafka locally via:

docker-compose.yml

docker-compose.yml

  zookeeper:
    image: 'bitnami/zookeeper:latest'
    ports:
      - 2181:2181
    environment:
      - ALLOW_ANONYMOUS_LOGIN=yes
  kafka:
    image: 'bitnami/kafka:latest'
    ports:
      - 9092:9092
    environment:
      - KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
      - ALLOW_PLAINTEXT_LISTENER=yes
      - KAFKA_ADVERTISED_PORT=9092
      - KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://localhost:9092

我的 Spring Boot 应用程序运行于:

My Spring Boot application is run with:

application.yml:

application.yml:

spring:
  application:
    name: testkafka
  kafka:
    bootstrap-servers: localhost:9092

server:
  port: 8080

当我运行它并尝试发送到 kafka 上的主题时，我得到:

When I run it and it tries to send to a topic on kafka, I get:

org.springframework.kafka.KafkaException: Reply timed out
    at org.springframework.kafka.requestreply.ReplyingKafkaTemplate.lambda$sendAndReceive$0(ReplyingKafkaTemplate.java:196) ~[spring-kafka-2.1.10.RELEASE.jar:2.1.10.RELEASE]
    at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) ~[spring-context-5.0.10.RELEASE.jar:5.0.10.RELEASE]
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[na:na]
    at java.base/java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:264) ~[na:na]
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java) ~[na:na]
    at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) ~[na:na]
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[na:na]
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[na:na]
    at java.base/java.lang.Thread.run(Thread.java:834) ~[na:na]

如果我从 docker 容器内运行 spring boot(使用一个 docker compose 文件)，那么它确实工作:

If I run the spring boot from inside a docker container (using one docker compose file), then it does work:

在同一个 Compose 中运行两者:

Running Both in the Same Compose:

version: "3.0"
services:
  service1:
    build: ./Service
    ports:
      - 8080:8080
      - 5005:5005
    links:
      - zookeeper
      - kafka
  zookeeper:
    image: 'bitnami/zookeeper:latest'
    ports:
      - 2181:2181
    environment:
      - ALLOW_ANONYMOUS_LOGIN=yes
  kafka:
    image: 'bitnami/kafka:latest'
    ports:
      - 9092:9092
    environment:
      - KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
      - ALLOW_PLAINTEXT_LISTENER=yes
      - KAFKA_ADVERTISED_PORT=9092
      - KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://localhost:9092

如何让 kafka 容器允许来自外部/docker 的连接?

How can I get the kafka container to allow connections from outside itself/docker?

尝试了这些更改:

kafka:
  image: 'bitnami/kafka:latest'
  ports:
    - 9092:9092
  environment:
    - KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
    - ALLOW_PLAINTEXT_LISTENER=yes
    - KAFKA_ADVERTISED_PORT=9092
    - KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://kafka:9092

和:

spring:
  application:
    name: testkafka
  kafka:
    bootstrap-servers: kafka:9092

server:
  port: 8080

这仍然超时

推荐答案

如果我从 docker 容器内运行 spring boot(使用一个 docker compose 文件)，那么它确实有效

If I run the spring boot from inside a docker container (using one docker compose file), then it does work

实际上它不应该工作.Kafka 未作为应用程序的一部分运行，因此本节不指向 Kafka 容器.

It shouldn't work, actually. Kafka isn't running as part of the application, so this section is not pointing at the Kafka container.

kafka:
    bootstrap-servers: localhost:9092

它需要是 Docker 网络中的 kafka:9092.

It would need to be kafka:9092 within the Docker network.

和 ... bootstrap-servers: kafka:9092 ... 仍然超时

and ... bootstrap-servers: kafka:9092 ... still times out

那是因为你还有 KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://localhost:9092

在 Docker 网络内部和外部，通过指定 KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://localhost:9092，您是说您的客户端以 localhost:9092<接收 Kafka 引导程序连接/code>，它只能在 Docker 网络之外工作，因为您已经进行了端口转发并且您的容器在 localhost 上运行，但是，在 Docker 网络内部，如上所述，localhost 意味着 那个应用程序容器，而不是代理.

Both inside and outside of Docker network, by specifying KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://localhost:9092, you're saying that your clients receive the Kafka bootstrap connection as localhost:9092, which would work outside of the Docker network only because you have made both a port forwarding and your container is running on localhost, however, inside the Docker network, as mentioned, localhost would mean that application container, not the broker.

解决方案是通过该属性创建两个端口映射，正如 这篇博文

The solution would be to create two port mappings via that property, as discussed in length by this blog post

另外，Confluent 提供了一个完全配置的 Docker Compose，并带有适当的映射，可以在 Docker 内部和外部工作

Plus, Confluent provides a fully configured Docker Compose with appropriate mappings that'll work inside and out of Docker

    ports:
      - "9092:9092"
      - "29092:29092"
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
      ALLOW_PLAINTEXT_LISTENER: "yes"
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      KAFKA_LISTENERS: PLAINTEXT://:9092,PLAINTEXT_HOST://0.0.0.0:29092
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092,PLAINTEXT_HOST://localhost:29092

对于Docker网络内的应用，使用kafka:9092，对于外部的应用，使用localhost:29092

For applications within Docker network, use kafka:9092, for applications outside, use localhost:29092

EDIT bitnami 配置变量已更改.请参阅自述文件的这一部分

EDIT The bitnami config variables have changed. Refer this section of the README

https://github.com/bitnami/bitnami-docker-kafka/blob/master/README.md#accessing-kafka-with-internal-and-external-clients

这篇关于无法从容器外运行的 Spring Boot 应用程序连接到容器中运行的 Kafka的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！