Kafka Connect java.lang.NoSuchMethodError:com.google.common.collect.Sets $ SetView.iterator()Lcom/google/common/collect/UnmodifiableIterator; [英] Kafka Connect java.lang.NoSuchMethodError: com.google.common.collect.Sets$SetView.iterator()Lcom/google/common/collect/UnmodifiableIterator;
问题描述
我正在尝试在AWS实例上设置kafka-connect-cassandra.
I am trying to setup kafka-connect-cassandra on an AWS instance.
我在connect-avro-distributed.properties
文件中设置了plugin.path
:
plugin.path=/home/ubuntu/kafka_2.11-1.0.0/plugins
我在其中有kafka-connect-cassandra-1.0.0-1.0.0-all.jar
/home/ubuntu/kafka_2.11-1.0.0/plugins/lib
这是回溯:
[2018-02-18 10:28:33,268] INFO Kafka Connect distributed worker initializing ... (org.apache.kafka.connect.cli.ConnectDistributed:60)
[2018-02-18 10:28:33,278] INFO WorkerInfo values:
jvm.args = -Xmx256M, -XX:+UseG1GC, -XX:MaxGCPauseMillis=20, -XX:InitiatingHeapOccupancyPercent=35, -XX:+ExplicitGCInvokesConcurrent, -Djava.awt.headless=true, -Dcom.sun.management.jmxremote, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.ssl=false, -Dkafka.logs.dir=/var/log/kafka, -Dlog4j.configuration=file:/etc/kafka/connect-log4j.properties
jvm.spec = Oracle Corporation, Java HotSpot(TM) 64-Bit Server VM, 1.8.0_151, 25.151-b12
jvm.classpath = /home/ubuntu/kafka_2.11-1.0.0/plugins:/usr/share/java/kafka/jackson-jaxrs-json-provider-2.9.1.jar:/usr/share/java/kafka/paranamer-2.7.jar:/usr/share/java/kafka/rocksdbjni-5.7.3.jar:/usr/share/java/kafka/jackson-jaxrs-base-2.9.1.jar:/usr/share/java/kafka/jetty-servlets-9.2.22.v20170606.jar:/usr/share/java/kafka/kafka-clients-1.0.0-cp1.jar:/usr/share/java/kafka/xz-1.5.jar:/usr/share/java/kafka/commons-lang3-3.1.jar:/usr/share/java/kafka/jetty-security-9.2.22.v20170606.jar:/usr/share/java/kafka/httpclient-4.5.2.jar:/usr/share/java/kafka/jackson-core-asl-1.9.13.jar:/usr/share/java/kafka/connect-json-1.0.0-cp1.jar:/usr/share/java/kafka/connect-runtime-1.0.0-cp1.jar:/usr/share/java/kafka/jersey-container-servlet-2.25.1.jar:/usr/share/java/kafka/commons-collections-3.2.1.jar:/usr/share/java/kafka/connect-transforms-1.0.0-cp1.jar:/usr/share/java/kafka/jersey-common-2.25.1.jar:/usr/share/java/kafka/zookeeper-3.4.10.jar:/usr/share/java/kafka/scala-library-2.11.11.jar:/usr/share/java/kafka/jackson-core-2.9.1.jar:/usr/share/java/kafka/argparse4j-0.7.0.jar:/usr/share/java/kafka/maven-artifact-3.5.0.jar:/usr/share/java/kafka/kafka_2.11-1.0.0-cp1-test-sources.jar:/usr/share/java/kafka/commons-codec-1.9.jar:/usr/share/java/kafka/kafka-connect-cassandra-1.0.0-1.0.0-all.jar:/usr/share/java/kafka/httpcore-4.4.4.jar:/usr/share/java/kafka/hk2-utils-2.5.0-b32.jar:/usr/share/java/kafka/connect-api-1.0.0-cp1.jar:/usr/share/java/kafka/javassist-3.21.0-GA.jar:/usr/share/java/kafka/kafka_2.11-1.0.0-cp1-sources.jar:/usr/share/java/kafka/support-metrics-client-4.0.0.jar:/usr/share/java/kafka/commons-compress-1.8.1.jar:/usr/share/java/kafka/kafka_2.11-1.0.0-cp1-scaladoc.jar:/usr/share/java/kafka/javax.servlet-api-3.1.0.jar:/usr/share/java/kafka/jersey-media-jaxb-2.25.1.jar:/usr/share/java/kafka/kafka-streams-1.0.0-cp1.jar:/usr/share/java/kafka/zkclient-0.10.jar:/usr/share/java/kafka/hk2-locator-2.5.0-b32.jar:/usr/share/java/kafka/slf4j-api-1.7.25.jar:/usr/share/java/kafka/support-metrics-common-4.0.0.jar:/usr/share/java/kafka/kafka.jar:/usr/share/java/kafka/jersey-server-2.25.1.jar:/usr/share/java/kafka/jackson-module-jaxb-annotations-2.9.1.jar:/usr/share/java/kafka/jetty-io-9.2.22.v20170606.jar:/usr/share/java/kafka/kafka-log4j-appender-1.0.0-cp1.jar:/usr/share/java/kafka/avro-1.8.2.jar:/usr/share/java/kafka/jackson-annotations-2.9.1.jar:/usr/share/java/kafka/guava-20.0.jar:/usr/share/java/kafka/hk2-api-2.5.0-b32.jar:/usr/share/java/kafka/lz4-java-1.4.jar:/usr/share/java/kafka/reflections-0.9.11.jar:/usr/share/java/kafka/commons-digester-1.8.1.jar:/usr/share/java/kafka/slf4j-log4j12-1.7.25.jar:/usr/share/java/kafka/jersey-client-2.25.1.jar:/usr/share/java/kafka/commons-lang3-3.5.jar:/usr/share/java/kafka/jackson-mapper-asl-1.9.13.jar:/usr/share/java/kafka/javax.annotation-api-1.2.jar:/usr/share/java/kafka/snappy-java-1.1.4.jar:/usr/share/java/kafka/javax.inject-2.5.0-b32.jar:/usr/share/java/kafka/jackson-databind-2.9.1.jar:/usr/share/java/kafka/jetty-http-9.2.22.v20170606.jar:/usr/share/java/kafka/kafka-streams-examples-1.0.0-cp1.jar:/usr/share/java/kafka/plexus-utils-3.0.24.jar:/usr/share/java/kafka/metrics-core-2.2.0.jar:/usr/share/java/kafka/connect-file-1.0.0-cp1.jar:/usr/share/java/kafka/kafka-tools-1.0.0-cp1.jar:/usr/share/java/kafka/jersey-guava-2.25.1.jar:/usr/share/java/kafka/commons-logging-1.2.jar:/usr/share/java/kafka/jetty-server-9.2.22.v20170606.jar:/usr/share/java/kafka/validation-api-1.1.0.Final.jar:/usr/share/java/kafka/jetty-continuation-9.2.22.v20170606.jar:/usr/share/java/kafka/osgi-resource-locator-1.0.1.jar:/usr/share/java/kafka/httpmime-4.5.2.jar:/usr/share/java/kafka/log4j-1.2.17.jar:/usr/share/java/kafka/jopt-simple-5.0.4.jar:/usr/share/java/kafka/kafka_2.11-1.0.0-cp1-javadoc.jar:/usr/share/java/kafka/javax.ws.rs-api-2.0.1.jar:/usr/share/java/kafka/jersey-container-servlet-core-2.25.1.jar:/usr/share/java/kafka/jetty-servlet-9.2.22.v20170606.jar:/usr/share/java/kafka/jetty-util-9.2.22.v20170606.jar:/usr/share/java/kafka/commons-validator-1.4.1.jar:/usr/share/java/kafka/kafka_2.11-1.0.0-cp1.jar:/usr/share/java/kafka/javax.inject-1.jar:/usr/share/java/kafka/commons-beanutils-1.8.3.jar:/usr/share/java/kafka/javassist-3.20.0-GA.jar:/usr/share/java/kafka/aopalliance-repackaged-2.5.0-b32.jar:/usr/share/java/kafka/kafka_2.11-1.0.0-cp1-test.jar:/usr/share/java/confluent-common/zookeeper-3.4.10.jar:/usr/share/java/confluent-common/common-metrics-4.0.0.jar:/usr/share/java/confluent-common/build-tools-4.0.0.jar:/usr/share/java/confluent-common/zkclient-0.10.jar:/usr/share/java/confluent-common/slf4j-api-1.7.25.jar:/usr/share/java/confluent-common/common-utils-4.0.0.jar:/usr/share/java/confluent-common/netty-3.10.5.Final.jar:/usr/share/java/confluent-common/common-config-4.0.0.jar:/usr/share/java/confluent-common/jline-0.9.94.jar:/usr/share/java/confluent-common/log4j-1.2.17.jar:/usr/share/java/kafka-serde-tools/kafka-json-serializer-4.0.0.jar:/usr/share/java/kafka-serde-tools/paranamer-2.7.jar:/usr/share/java/kafka-serde-tools/xz-1.5.jar:/usr/share/java/kafka-serde-tools/jackson-core-asl-1.9.13.jar:/usr/share/java/kafka-serde-tools/jackson-core-2.9.1.jar:/usr/share/java/kafka-serde-tools/kafka-connect-avro-converter-4.0.0.jar:/usr/share/java/kafka-serde-tools/commons-compress-1.8.1.jar:/usr/share/java/kafka-serde-tools/avro-1.8.2.jar:/usr/share/java/kafka-serde-tools/jackson-annotations-2.9.1.jar:/usr/share/java/kafka-serde-tools/kafka-schema-registry-client-4.0.0.jar:/usr/share/java/kafka-serde-tools/kafka-avro-serializer-4.0.0.jar:/usr/share/java/kafka-serde-tools/jackson-mapper-asl-1.9.13.jar:/usr/share/java/kafka-serde-tools/jackson-databind-2.9.1.jar:/usr/share/java/kafka-serde-tools/snappy-java-1.1.1.3.jar:/usr/bin/../share/java/kafka/jackson-jaxrs-json-provider-2.9.1.jar:/usr/bin/../share/java/kafka/paranamer-2.7.jar:/usr/bin/../share/java/kafka/rocksdbjni-5.7.3.jar:/usr/bin/../share/java/kafka/jackson-jaxrs-base-2.9.1.jar:/usr/bin/../share/java/kafka/jetty-servlets-9.2.22.v20170606.jar:/usr/bin/../share/java/kafka/kafka-clients-1.0.0-cp1.jar:/usr/bin/../share/java/kafka/xz-1.5.jar:/usr/bin/../share/java/kafka/commons-lang3-3.1.jar:/usr/bin/../share/java/kafka/jetty-security-9.2.22.v20170606.jar:/usr/bin/../share/java/kafka/httpclient-4.5.2.jar:/usr/bin/../share/java/kafka/jackson-core-asl-1.9.13.jar:/usr/bin/../share/java/kafka/connect-json-1.0.0-cp1.jar:/usr/bin/../share/java/kafka/connect-runtime-1.0.0-cp1.jar:/usr/bin/../share/java/kafka/jersey-container-servlet-2.25.1.jar:/usr/bin/../share/java/kafka/commons-collections-3.2.1.jar:/usr/bin/../share/java/kafka/connect-transforms-1.0.0-cp1.jar:/usr/bin/../share/java/kafka/jersey-common-2.25.1.jar:/usr/bin/../share/java/kafka/zookeeper-3.4.10.jar:/usr/bin/../share/java/kafka/scala-library-2.11.11.jar:/usr/bin/../share/java/kafka/jackson-core-2.9.1.jar:/usr/bin/../share/java/kafka/argparse4j-0.7.0.jar:/usr/bin/../share/java/kafka/maven-artifact-3.5.0.jar:/usr/bin/../share/java/kafka/kafka_2.11-1.0.0-cp1-test-sources.jar:/usr/bin/../share/java/kafka/commons-codec-1.9.jar:/usr/bin/../share/java/kafka/kafka-connect-cassandra-1.0.0-1.0.0-all.jar:/usr/bin/../share/java/kafka/httpcore-4.4.4.jar:/usr/bin/../share/java/kafka/hk2-utils-2.5.0-b32.jar:/usr/bin/../share/java/kafka/connect-api-1.0.0-cp1.jar:/usr/bin/../share/java/kafka/javassist-3.21.0-GA.jar:/usr/bin/../share/java/kafka/kafka_2.11-1.0.0-cp1-sources.jar:/usr/bin/../share/java/kafka/support-metrics-client-4.0.0.jar:/usr/bin/../share/java/kafka/commons-compress-1.8.1.jar:/usr/bin/../share/java/kafka/kafka_2.11-1.0.0-cp1-scaladoc.jar:/usr/bin/../share/java/kafka/javax.servlet-api-3.1.0.jar:/usr/bin/../share/java/kafka/jersey-media-jaxb-2.25.1.jar:/usr/bin/../share/java/kafka/kafka-streams-1.0.0-cp1.jar:/usr/bin/../share/java/kafka/zkclient-0.10.jar:/usr/bin/../share/java/kafka/hk2-locator-2.5.0-b32.jar:/usr/bin/../share/java/kafka/slf4j-api-1.7.25.jar:/usr/bin/../share/java/kafka/support-metrics-common-4.0.0.jar:/usr/bin/../share/java/kafka/kafka.jar:/usr/bin/../share/java/kafka/jersey-server-2.25.1.jar:/usr/bin/../share/java/kafka/jackson-module-jaxb-annotations-2.9.1.jar:/usr/bin/../share/java/kafka/jetty-io-9.2.22.v20170606.jar:/usr/bin/../share/java/kafka/kafka-log4j-appender-1.0.0-cp1.jar:/usr/bin/../share/java/kafka/avro-1.8.2.jar:/usr/bin/../share/java/kafka/jackson-annotations-2.9.1.jar:/usr/bin/../share/java/kafka/guava-20.0.jar:/usr/bin/../share/java/kafka/hk2-api-2.5.0-b32.jar:/usr/bin/../share/java/kafka/lz4-java-1.4.jar:/usr/bin/../share/java/kafka/reflections-0.9.11.jar:/usr/bin/../share/java/kafka/commons-digester-1.8.1.jar:/usr/bin/../share/java/kafka/slf4j-log4j12-1.7.25.jar:/usr/bin/../share/java/kafka/jersey-client-2.25.1.jar:/usr/bin/../share/java/kafka/commons-lang3-3.5.jar:/usr/bin/../share/java/kafka/jackson-mapper-asl-1.9.13.jar:/usr/bin/../share/java/kafka/javax.annotation-api-1.2.jar:/usr/bin/../share/java/kafka/snappy-java-1.1.4.jar:/usr/bin/../share/java/kafka/javax.inject-2.5.0-b32.jar:/usr/bin/../share/java/kafka/jackson-databind-2.9.1.jar:/usr/bin/../share/java/kafka/jetty-http-9.2.22.v20170606.jar:/usr/bin/../share/java/kafka/kafka-streams-examples-1.0.0-cp1.jar:/usr/bin/../share/java/kafka/plexus-utils-3.0.24.jar:/usr/bin/../share/java/kafka/metrics-core-2.2.0.jar:/usr/bin/../share/java/kafka/connect-file-1.0.0-cp1.jar:/usr/bin/../share/java/kafka/kafka-tools-1.0.0-cp1.jar:/usr/bin/../share/java/kafka/jersey-guava-2.25.1.jar:/usr/bin/../share/java/kafka/commons-logging-1.2.jar:/usr/bin/../share/java/kafka/jetty-server-9.2.22.v20170606.jar:/usr/bin/../share/java/kafka/validation-api-1.1.0.Final.jar:/usr/bin/../share/java/kafka/jetty-continuation-9.2.22.v20170606.jar:/usr/bin/../share/java/kafka/osgi-resource-locator-1.0.1.jar:/usr/bin/../share/java/kafka/httpmime-4.5.2.jar:/usr/bin/../share/java/kafka/log4j-1.2.17.jar:/usr/bin/../share/java/kafka/jopt-simple-5.0.4.jar:/usr/bin/../share/java/kafka/kafka_2.11-1.0.0-cp1-javadoc.jar:/usr/bin/../share/java/kafka/javax.ws.rs-api-2.0.1.jar:/usr/bin/../share/java/kafka/jersey-container-servlet-core-2.25.1.jar:/usr/bin/../share/java/kafka/jetty-servlet-9.2.22.v20170606.jar:/usr/bin/../share/java/kafka/jetty-util-9.2.22.v20170606.jar:/usr/bin/../share/java/kafka/commons-validator-1.4.1.jar:/usr/bin/../share/java/kafka/kafka_2.11-1.0.0-cp1.jar:/usr/bin/../share/java/kafka/javax.inject-1.jar:/usr/bin/../share/java/kafka/commons-beanutils-1.8.3.jar:/usr/bin/../share/java/kafka/javassist-3.20.0-GA.jar:/usr/bin/../share/java/kafka/aopalliance-repackaged-2.5.0-b32.jar:/usr/bin/../share/java/kafka/kafka_2.11-1.0.0-cp1-test.jar:/usr/bin/../share/java/confluent-support-metrics/*:/usr/share/java/confluent-support-metrics/*
os.spec = Linux, amd64, 4.4.0-1049-aws
os.vcpus = 2
(org.apache.kafka.connect.runtime.WorkerInfo:71)
[2018-02-18 10:28:33,279] INFO Scanning for plugin classes. This might take a moment ... (org.apache.kafka.connect.cli.ConnectDistributed:69)
[2018-02-18 10:28:33,290] INFO Loading plugin from: /home/ubuntu/kafka_2.11-1.0.0/plugins/lib (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:179)
Exception in thread "main" java.lang.NoSuchMethodError: com.google.common.collect.Sets$SetView.iterator()Lcom/google/common/collect/UnmodifiableIterator;
at org.reflections.Reflections.expandSuperTypes(Reflections.java:380)
at org.reflections.Reflections.<init>(Reflections.java:126)
at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.scanPluginPath(DelegatingClassLoader.java:258)
at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.scanUrlsAndAddPlugins(DelegatingClassLoader.java:201)
at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.registerPlugin(DelegatingClassLoader.java:193)
at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.initLoaders(DelegatingClassLoader.java:153)
at org.apache.kafka.connect.runtime.isolation.Plugins.<init>(Plugins.java:47)
at org.apache.kafka.connect.cli.ConnectDistributed.main(ConnectDistributed.java:70)
jvm.classpath
中的第一项是我在plugin/lib
中拥有kafka-connect-cassandra.jar
的位置.
The first entry in jvm.classpath
is the location where I have kafka-connect-cassandra.jar
, inside plugin/lib
.
guava.jars:
这些是番石榴罐子的路径.我应该在哪里放置kafka-connect-cassandra.jar还是应该删除其中的所有罐子?
These are the paths of guava jars. Where should I place the kafka-connect-cassandra.jar or should I just remove any of these jars?
/usr/share/java/kafka-connect-elasticsearch/guava-18.0.jar
/usr/share/java/kafka-connect-storage-common/guava-14.0.1.jar
/usr/share/java/kafka/guava-20.0.jar
/home/ubuntu/cassandra/apache-cassandra-3.11.1/lib/guava-18.0.jar
请帮助我.
推荐答案
Connect框架当前带有guava-20.0.jar
以便使用Reflections并能够扫描plugin.path
中的Connect插件.
The Connect framework currently carries guava-20.0.jar
in order to use Reflections and be able to scan your plugin.path
for Connect plugins.
Connect框架的依赖项添加到CLASSPATH中.
Connect framework's dependencies are added to your CLASSPATH if you are starting Connect using one of its bin scripts: connect-distributed
or connect-standalone
.
但是,以某种方式,旧版本的guava首先驻留在CLASSPATH中(可能是因为您已将CLASSPATH设置为包括这样的依赖项),然后Connect无法使用guava版本plugin.path
.
However, somehow an older version of guava is residing first in your CLASSPATH (maybe because you've set the CLASSPATH to include such a dependency), and then Connect is unable to scan your plugin.path
with a guava version < 20
. That's what your exception indicates. That Connect can't scan the plugin.path
with the version of guava it finds in your CLASSPATH.
您的plugin.path
应该包含您的插件,每个插件都在其自己的目录中.例如,类似于上面的内容:
Your plugin.path
should contain your plugins, each on its own directory. For example, similarly to what you have above:
plugin.path=/home/ubuntu/connect
与
/home/ubuntu/connect/kafka_2.11-1.0.0/
和
/home/ubuntu/connect/kafka-connect-cassandra-1.0.0-1.0.0-all.jar
根据您的示例,上面的布局用于说明插件不应包含其他插件,但是不建议将plugin.path
设置为用户的主目录.
The layout above is used to make the point that plugins should not contain other plugins according to your example, but it's not recommended to set your plugin.path
to a user's home directory.
这篇关于Kafka Connect java.lang.NoSuchMethodError:com.google.common.collect.Sets $ SetView.iterator()Lcom/google/common/collect/UnmodifiableIterator;的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!