PubSub Kafka Connect 节点连接文件异常结束 [英] PubSub Kafka Connect node connection end of file exception

问题描述

使用以下命令运行 PubSub Kafka 连接:

While running PubSub Kafka connect using the command:

.\bin\windows\connect-standalone.bat .\etc\kafka\WorkerConfig.properties
 .\etc\kafka\configSink.properties .\etc\kafka\configSource.properties

我收到此错误:

Sending metadata request {topics=[test]} to node -1
could not scan file META-INF/MANIFEST.MF in url file:/C:/confluent-3.3.0/bin/../share/java/kafka-serde-tools/commons-compress-1.8.1.jar with scanner SubTypesScanner
could not scan file META-INF/MANIFEST.MF in url file:/C:/confluent-3.3.0/bin/../share/java/kafka-serde-tools/commons-compress-1.8.1.jar with scanner TypeAnnotationsScanner
could not scan file META-INF/LICENSE.txt in url file:/C:/confluent-3.3.0/bin/../share/java/kafka-serde-tools/commons-compress-1.8.1.jar with scanner SubTypesScanner
could not scan file META-INF/LICENSE.txt in url file:/C:/confluent-3.3.0/bin/../share/java/kafka-serde-tools/commons-compress-1.8.1.jar with scanner TypeAnnotationsScanner
could not scan file META-INF/NOTICE.txt in url file:/C:/confluent-3.3.0/bin/../share/java/kafka-serde-tools/commons-compress-1.8.1.jar with scanner SubTypesScanner
could not scan file META-INF/NOTICE.txt in url file:/C:/confluent-3.3.0/bin/../share/java/kafka-serde-tools/commons-compress-1.8.1.jar with scanner TypeAnnotationsScanner
Connection with localhost/127.0.0.1 disconnected
java.io.EOFException
    at org.apache.kafka.common.network.NetworkReceive.readFromReadableChannel(NetworkReceive.java:83)
    at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:71)
    at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:154)
    at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:135)
    at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:323)
    at org.apache.kafka.common.network.Selector.poll(Selector.java:283)
    at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:260)
    at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.clientPoll(ConsumerNetworkClient.java:360)
    at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:224)
    at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:192)
    at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:163)
    at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureCoordinatorReady(AbstractCoordinator.java:181)
    at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureActiveGroup(AbstractCoordinator.java:229)
    at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.ensurePartitionAssignment(ConsumerCoordinator.java:366)
    at org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce(KafkaConsumer.java:975)
    at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:938)
    at org.apache.kafka.connect.runtime.WorkerSinkTask.pollConsumer(WorkerSinkTask.java:316)
    at org.apache.kafka.connect.runtime.WorkerSinkTask.poll(WorkerSinkTask.java:222)
    at org.apache.kafka.connect.runtime.WorkerSinkTask.iteration(WorkerSinkTask.java:170)
    at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:142)
    at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:140)
    at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:175)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
Node -1 disconnected.
Bootstrap broker localhost:2181 disconnected
Cancelled GROUP_COORDINATOR request ClientRequest(expectResponse=true, callback=org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient$RequestFutureCompletionHandler@643b72cb, request=RequestSend(header={api_key=10,api_version=0,correlation_id=0,client_id=consumer-1}, body={group_id=connect-CPSConnectorSink}), createdTimeMs=1504105687600, sendTimeMs=1504105687724) with correlation id 0 due to node -1 being disconnected
Give up sending metadata request since no node is available

我正在运行本地 kafka 代理和本地动物园管理员.任何有关如何解决此问题的帮助都会很棒！

I am running a local kafka broker and local zookeeper. Any help on how to fix this issue would be great!

属性文件此处

推荐答案

所以，在尝试了各种修复之后，我发现我使用了错误的 localhost 端口号.Zookeeper 使用了端口 2181，而 Kafka Server 使用了端口 9092.所以，我在 WorkerConfig.properties 中的原始属性是:

So, after trying various fixes, I found out that I was using the wrong port number for localhost. The port 2181 was being used by Zookeeper while port 9092 was being used by Kafka Server. So, my original property in WorkerConfig.properties was:

bootstrap.servers=localhost:2181

什么时候该

bootstrap.server=localhost:9092

因此，故事的寓意是确保您在连接器属性文件中使用正确的 Kafka 服务器主机和端口号，我的是 WorkerConfig.这显示在此处.这似乎是常识，但我在尝试修复其他错误时更改了端口号，但在不知不觉中添加了另一个错误.因此，如果您收到上述错误，请检查您的 bootstrap.server 属性.

So, moral of the story is to make sure you are using the correct host and port number of your Kafka servers in your connector properties file, mine being WorkerConfig. This is shown here. This may seem like common sense, but I changed the port number while trying to fix other errors, but unknowingly added another error. So, if you recieve the above error, check your bootstrap.server property.

这篇关于PubSub Kafka Connect 节点连接文件异常结束的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！