Debezium：无法在Kafka-Connect Docker容器上设置观察程序。提取具有关联ID 2的元数据时出错... LEADER_NOT_AVAILABLE [英] Debezium: Unable to setup watcher on Kafka-Connect Docker container. Error while fetching metadata with correlation id 2 ... LEADER_NOT_AVAILABLE

问题描述

问题：

如何设置观察者以实时观察Kafka-Connect流，因为对数据库进行了更改？在获取具有相关性ID 2的元数据时，出现错误：{dbhistory.Test.Posts = LEADER_NOT_AVAILABLE} 。是什么原因导致该问题以及如何解决？

How do I setup watcher to watch a Kafka-Connect stream in real-time as changes are made to the database? I am getting Error while fetching metadata with correlation id 2 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE}. What is causing the problem and how do I fix it?

上下文：

此问题与以下内容有关： Debezium如何正确向Kafka Connect注册SqlServer连接器-连接被拒绝和
https://github.com/debezium/debezium-examples/blob/master/tutorial/README.md#using-sql-server

This question is related to: Debezium How do I correctly register the SqlServer connector with Kafka Connect - connection refused and https://github.com/debezium/debezium-examples/blob/master/tutorial/README.md#using-sql-server

在Windows 10中，我使用以下命令在各自独立的Docker容器中设置了Zookeeper，Kafka和Kafka-Connector：

In Windows 10, I have setup Zookeeper, Kafka, and Kafka-Connector all in their own separate Docker containers using the following commands:

docker run -it --rm --name zookeeper -p 2181:2181 -p 2888:2888 -p 3888:3888 debezium/zookeeper:1.1
docker run -it --rm --name kafka -p 9092:9092 -e KAFKA_ADVERTISED_HOST_NAME: kafka --link zookeeper:zookeeper debezium/kafka:1.1
docker run -it --rm --name connect -p 8083:8083 -e GROUP_ID=1 -e CONFIG_STORAGE_TOPIC=my_connect_configs -e OFFSET_STORAGE_TOPIC=my_connect_offsets -e STATUS_STORAGE_TOPIC=my_connect_statuses --link zookeeper:zookeeper --link kafka:kafka debezium/connect:1.1

I然后在Docker容器外部，将Kafka连接器设置为在本地主机上运行的SQL Server实例。日志显示连接器可以连接到数据库并获取CDC数据。到目前为止，我还可以。

I then setup a Kafka Connector to an instance of SQL Server that is running on the localhost, outside of a Docker container. The logs show that the connector can connect to the database and grab the CDC data. Up until this point I am OK.

但是，当我尝试创建观察者时，请使用以下命令：

However, when I try to create a watcher, using the following command:

docker run -it --rm --name watcher --link zookeeper:zookeeper --link kafka:kafka debezium/kafka:1.1 

我收到以下错误。

watch-topic -a -k dbhistory.Test.Posts
WARNING: Using default BROKER_ID=1, which is valid only for non-clustered installations.
Using ZOOKEEPER_CONNECT=172.17.0.2:2181
Using KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://172.17.0.5:9092
Using KAFKA_BROKER=172.17.0.3:9092
Contents of topic dbhistory.Test.Posts:
[2020-04-08 15:18:39,373] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 2 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:41:20,501] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:41:20,605] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:41:20,807] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:41:21,262] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:41:22,120] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:41:23,330] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:41:24,189] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:41:59,826] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:04,051] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:07,121] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:08,024] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:09,354] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11570 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:09,458] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11572 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:09,571] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11574 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:09,677] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11576 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:09,786] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11578 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:09,895] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11580 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:10,002] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11582 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:10,112] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11584 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:10,220] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11586 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:10,326] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11588 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:10,435] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11590 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:10,537] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11592 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:10,648] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11594 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:10,751] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11596 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:10,854] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11598 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 17:08:48,268] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 17:09:55,602] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)

推荐答案

您可以尝试在docker run中添加--network = host吗？

Can you try once adding --network=host in the docker run.

这篇关于Debezium：无法在Kafka-Connect Docker容器上设置观察程序。提取具有关联ID 2的元数据时出错... LEADER_NOT_AVAILABLE的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！