无法从容器外部运行的Spring Boot应用连接到容器中运行的Kafka [英] Unable to connect to Kafka run in container from Spring Boot app run outside container

问题描述

我正在通过以下方式在本地运行kafka：

I'm running kafka locally via:

docker-compose.yml

docker-compose.yml

  zookeeper:
    image: 'bitnami/zookeeper:latest'
    ports:
      - 2181:2181
    environment:
      - ALLOW_ANONYMOUS_LOGIN=yes
  kafka:
    image: 'bitnami/kafka:latest'
    ports:
      - 9092:9092
    environment:
      - KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
      - ALLOW_PLAINTEXT_LISTENER=yes
      - KAFKA_ADVERTISED_PORT=9092
      - KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://localhost:9092

我的Spring Boot应用程序运行于：

My Spring Boot application is run with:

application.yml：

application.yml:

spring:
  application:
    name: testkafka
  kafka:
    bootstrap-servers: localhost:9092

server:
  port: 8080

当我运行它并尝试发送有关kafka的主题时，我得到：

When I run it and it tries to send to a topic on kafka, I get:

org.springframework.kafka.KafkaException: Reply timed out
    at org.springframework.kafka.requestreply.ReplyingKafkaTemplate.lambda$sendAndReceive$0(ReplyingKafkaTemplate.java:196) ~[spring-kafka-2.1.10.RELEASE.jar:2.1.10.RELEASE]
    at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) ~[spring-context-5.0.10.RELEASE.jar:5.0.10.RELEASE]
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[na:na]
    at java.base/java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:264) ~[na:na]
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java) ~[na:na]
    at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) ~[na:na]
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[na:na]
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[na:na]
    at java.base/java.lang.Thread.run(Thread.java:834) ~[na:na]

如果我从docker容器内部运行spring boot（使用一个docker compose文件），那么它确实 >工作：

If I run the spring boot from inside a docker container (using one docker compose file), then it does work:

同时运行两个文件：

version: "3.0"
services:
  service1:
    build: ./Service
    ports:
      - 8080:8080
      - 5005:5005
    links:
      - zookeeper
      - kafka
  zookeeper:
    image: 'bitnami/zookeeper:latest'
    ports:
      - 2181:2181
    environment:
      - ALLOW_ANONYMOUS_LOGIN=yes
  kafka:
    image: 'bitnami/kafka:latest'
    ports:
      - 9092:9092
    environment:
      - KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
      - ALLOW_PLAINTEXT_LISTENER=yes
      - KAFKA_ADVERTISED_PORT=9092
      - KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://localhost:9092

如何获取kafka容器以允许来自外部的本身/泊坞窗连接？

How can I get the kafka container to allow connections from outside itself/docker?

编辑：尝试了以下更改：

kafka:
  image: 'bitnami/kafka:latest'
  ports:
    - 9092:9092
  environment:
    - KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
    - ALLOW_PLAINTEXT_LISTENER=yes
    - KAFKA_ADVERTISED_PORT=9092
    - KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://kafka:9092

并且：

spring:
  application:
    name: testkafka
  kafka:
    bootstrap-servers: kafka:9092

server:
  port: 8080

这仍然超时

推荐答案

如果我运行spring boot从一个docker容器内部（使用一个docker compose文件），然后它确实起作用了

If I run the spring boot from inside a docker container (using one docker compose file), then it does work

它不起作用，启动莉莉Kafka并未作为应用程序的一部分运行，因此本节未针对Kafka容器。

It shouldn't work, actually. Kafka isn't running as part of the application, so this section is not pointing at the Kafka container.

kafka:
    bootstrap-servers: localhost:9092

在Docker网络中，它必须是 kafka：9092 。

It would need to be kafka:9092 within the Docker network.

通过指定 KAFKA_ADVERTISED_LISTENERS = PLAINTEXT：// localhost：9092 ，在Docker网络的内部和外部表示您的客户端以 localhost：9092 的身份接收Kafka引导程序连接，该连接仅在您进行端口转发且容器正在运行的情况下才能在Docker网络外部运行但是，在本地主机上，如上所述，在Docker网络内部，本地主机将表示那个应用程序容器，而不是代理。

Both inside and outside of Docker network, by specifying KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://localhost:9092, you're saying that your clients receive the Kafka bootstrap connection as localhost:9092, which would work outside of the Docker network only because you have made both a port forwarding and your container is running on localhost, however, inside the Docker network, as mentioned, localhost would mean that application container, not the broker.

通过该属性创建两个端口映射，如此博客文章详细讨论的那样

The solution would be to create two port mappings via that property, as discussed in length by this blog post

此外， Confluent提供了一个完整配置的Docker Compose ，该映射将在Docker内外

Plus, Confluent provides a fully configured Docker Compose with appropriate mappings that'll work inside and out of Docker

ports:
  - "9092:9092"
  - "29092:29092"
environment:
  KAFKA_BROKER_ID: 1
  KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
  ALLOW_PLAINTEXT_LISTENER: "yes"
  KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
  KAFKA_LISTENERS: PLAINTEXT://:9092,PLAINTEXT_HOST://:29092
  KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092,PLAINTEXT_HOST://localhost:29092

对于Docker网络中的应用程序，使用 kafka：9092 ，对于外部应用程序，使用 localhost：29092

For applications within Docker network, use kafka:9092, for applications outside, use localhost:29092

这篇关于无法从容器外部运行的Spring Boot应用连接到容器中运行的Kafka的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！