无法在分布式模式下运行 Kafka Connect - 尝试创建/查找主题“connect-offsets"时出错 [英] Not able to run Kafka Connect in distributed mode - Error while attempting to create/ find topics 'connect-offsets'

问题描述

[2017-08-31 10:15:20,715] WARN 配置提供了internal.key.converter"但不是已知的配置.(org.apache.kafka.clients.admin.AdminClientConfig:231) [2017-08-3110:15:20,715] 警告配置提供了status.storage.replication.factor"但不是已知的配置.(org.apache.kafka.clients.admin.AdminClientConfig:231)[2017-08-31 10:15:20,715] WARN 配置提供了internal.value.converter.schemas.enable"但不是已知配置(org.apache.kafka.clients.admin.AdminClientConfig:231)[2017-08-31 10:15:20,715] WARN 配置提供了value.converter.schemas.enable"但不是已知的配置.(org.apache.kafka.clients.admin.AdminClientConfig:231)[2017-08-31 10:15:20,715] WARN 配置提供了internal.value.converter"但不是已知的配置.(org.apache.kafka.clients.admin.AdminClientConfig:231) [2017-08-3110:15:20,715] 警告配置提供了offset.storage.replication.factor"但不是已知的配置.(org.apache.kafka.clients.admin.AdminClientConfig:231)[2017-08-31 10:15:20,716] WARN 配置提供了offset.storage.topic"，但不是已知的配置.(org.apache.kafka.clients.admin.AdminClientConfig:231) [2017-08-3110:15:20,716] 警告提供了配置value.converter"但不是已知的配置.(org.apache.kafka.clients.admin.AdminClientConfig:231) [2017-08-3110:15:20,716] WARN 提供了配置key.converter"，但不是已知的配置.(org.apache.kafka.clients.admin.AdminClientConfig:231) [2017-08-3110:15:20,768] INFO jetty-9.2.15.v20160210(org.eclipse.jetty.server.Server:327) 2017 年 8 月 31 日上午 10:15:21org.glassfish.jersey.internal.Errors logErrors 警告:以下已检测到警告:警告:(子)资源方法列出连接器org.apache.kafka.connect.runtime.rest.resources.ConnectorsResource包含空路径注释.警告:(子)资源方法在创建连接器org.apache.kafka.connect.runtime.rest.resources.ConnectorsResource包含空路径注释.警告:(子)资源方法列出连接器插件org.apache.kafka.connect.runtime.rest.resources.ConnectorPluginsResource包含空路径注释.警告:(子)资源方法服务器信息输入org.apache.kafka.connect.runtime.rest.resources.RootResource 包含空路径注释.

[2017-08-31 10:15:20,715] WARN The configuration 'internal.key.converter' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig:231) [2017-08-31 10:15:20,715] WARN The configuration 'status.storage.replication.factor' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig:231) [2017-08-31 10:15:20,715] WARN The configuration 'internal.value.converter.schemas.enable' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig:231) [2017-08-31 10:15:20,715] WARN The configuration 'value.converter.schemas.enable' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig:231) [2017-08-31 10:15:20,715] WARN The configuration 'internal.value.converter' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig:231) [2017-08-31 10:15:20,715] WARN The configuration 'offset.storage.replication.factor' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig:231) [2017-08-31 10:15:20,716] WARN The configuration 'offset.storage.topic' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig:231) [2017-08-31 10:15:20,716] WARN The configuration 'value.converter' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig:231) [2017-08-31 10:15:20,716] WARN The configuration 'key.converter' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig:231) [2017-08-31 10:15:20,768] INFO jetty-9.2.15.v20160210 (org.eclipse.jetty.server.Server:327) Aug 31, 2017 10:15:21 AM org.glassfish.jersey.internal.Errors logErrors WARNING: The following warnings have been detected: WARNING: The (sub)resource method listConnectors in org.apache.kafka.connect.runtime.rest.resources.ConnectorsResource contains empty path annotation. WARNING: The (sub)resource method createConnector in org.apache.kafka.connect.runtime.rest.resources.ConnectorsResource contains empty path annotation. WARNING: The (sub)resource method listConnectorPlugins in org.apache.kafka.connect.runtime.rest.resources.ConnectorPluginsResource contains empty path annotation. WARNING: The (sub)resource method serverInfo in org.apache.kafka.connect.runtime.rest.resources.RootResource contains empty path annotation.

[2017-08-31 10:15:21,142] INFO 开始o.e.j.s.ServletContextHandler@63f02de0{/,null,AVAILABLE}(org.eclipse.jetty.server.handler.ContextHandler:744) [2017-08-3110:15:21,149] 信息开始ServerConnector@61b06738{HTTP/1.1}{0.0.0.0:8083}(org.eclipse.jetty.server.ServerConnector:266) [2017-08-3110:15:21,149] 信息开始于 @13584 毫秒(org.eclipse.jetty.server.Server:379) [2017-08-31 10:15:21,150] 信息REST 服务器监听 http://172.28.193.126:8083/，广告 URLhttp://172.28.193.126:8083/(org.apache.kafka.connect.runtime.rest.RestServer:150) [2017-08-3110:15:21,150] INFO Kafka Connect 启动(org.apache.kafka.connect.runtime.Connect:55) [2017-08-3110:15:21,468] 错误 Herder 工作线程中未捕获的异常，退出:(org.apache.kafka.connect.runtime.distributed.DistributedHerder:206)org.apache.kafka.connect.errors.ConnectException: 时出错尝试在以下位置创建/查找主题connect-offsets"org.apache.kafka.connect.util.TopicAdmin.createTopics(TopicAdmin.java:245)在org.apache.kafka.connect.storage.KafkaOffsetBackingStore$1.run(KafkaOffsetBackingStore.java:99)在org.apache.kafka.connect.util.KafkaBasedLog.start(KafkaBasedLog.java:126)在org.apache.kafka.connect.storage.KafkaOffsetBackingStore.start(KafkaOffsetBackingStore.java:109)在 org.apache.kafka.connect.runtime.Worker.start(Worker.java:146) 在org.apache.kafka.connect.runtime.AbstractHerder.startServices(AbstractHerder.java:99)在org.apache.kafka.connect.runtime.distributed.DistributedHerder.run(DistributedHerder.java:194)在java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)在 java.util.concurrent.FutureTask.run(FutureTask.java:266) 在java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)在java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)在 java.lang.Thread.run(Thread.java:748) 引起的:java.util.concurrent.ExecutionException:org.apache.kafka.common.errors.UnknownServerException:服务器在处理请求时遇到意外错误org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45)在org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32)在org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:89)在org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:213)在org.apache.kafka.connect.util.TopicAdmin.createTopics(TopicAdmin.java:226)

[2017-08-31 10:15:21,142] INFO Started o.e.j.s.ServletContextHandler@63f02de0{/,null,AVAILABLE} (org.eclipse.jetty.server.handler.ContextHandler:744) [2017-08-31 10:15:21,149] INFO Started ServerConnector@61b06738{HTTP/1.1}{0.0.0.0:8083} (org.eclipse.jetty.server.ServerConnector:266) [2017-08-31 10:15:21,149] INFO Started @13584ms (org.eclipse.jetty.server.Server:379) [2017-08-31 10:15:21,150] INFO REST server listening at http://172.28.193.126:8083/, advertising URL http://172.28.193.126:8083/ (org.apache.kafka.connect.runtime.rest.RestServer:150) [2017-08-31 10:15:21,150] INFO Kafka Connect started (org.apache.kafka.connect.runtime.Connect:55) [2017-08-31 10:15:21,468] ERROR Uncaught exception in herder work thread, exiting: (org.apache.kafka.connect.runtime.distributed.DistributedHerder:206) org.apache.kafka.connect.errors.ConnectException: Error while attempting to create/find topic(s) 'connect-offsets' at org.apache.kafka.connect.util.TopicAdmin.createTopics(TopicAdmin.java:245) at org.apache.kafka.connect.storage.KafkaOffsetBackingStore$1.run(KafkaOffsetBackingStore.java:99) at org.apache.kafka.connect.util.KafkaBasedLog.start(KafkaBasedLog.java:126) at org.apache.kafka.connect.storage.KafkaOffsetBackingStore.start(KafkaOffsetBackingStore.java:109) at org.apache.kafka.connect.runtime.Worker.start(Worker.java:146) at org.apache.kafka.connect.runtime.AbstractHerder.startServices(AbstractHerder.java:99) at org.apache.kafka.connect.runtime.distributed.DistributedHerder.run(DistributedHerder.java:194) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.UnknownServerException: The server experienced an unexpected error when processing the request at org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45) at org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32) at org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:89) at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:213) at org.apache.kafka.connect.util.TopicAdmin.createTopics(TopicAdmin.java:226)

推荐答案

Kafka Connect 0.11.0.0 无法在 0.10.1.0 之前的 broker 上创建主题，因为这些较旧的 broker 中使用的协议不支持主题管理.

Kafka Connect 0.11.0.0 cannot create the topics on brokers older than 0.10.1.0, since the protocol used in those older brokers do not support topic management.

因此，如果您有一个较旧的代理，您应该手动创建主题或依赖代理自动创建.不幸的是，Kafka Connect 0.11.0.0 中的一个错误阻止了自动创建，并且因此，在 0.11.0.1 可用之前，您必须手动创建主题.

So, if you have an older broker, you should either manually create the topics or rely upon broker auto-creation. Unfortunately, a bug in Kafka Connect 0.11.0.0 prevents auto-creation, and so until 0.11.0.1 is available you will have to manually create the topics.

这篇关于无法在分布式模式下运行 Kafka Connect - 尝试创建/查找主题“connect-offsets"时出错的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！