KafkaSpout为log4j抛出NoClassDefFoundError [英] KafkaSpout throws NoClassDefFoundError for log4j
本文介绍了KafkaSpout为log4j抛出NoClassDefFoundError的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
由于某种原因,当我尝试在Storm集群上运行拓扑时出现以下错误:
For some reason I get the following error when I try to run my topology on a Storm cluster:
java.lang.NoClassDefFoundError: Could not initialize class org.apache.log4j.Log4jLoggerFactory
at org.apache.log4j.Logger.getLogger(Logger.java:39)
at kafka.utils.Logging$class.logger(Logging.scala:24)
at kafka.consumer.SimpleConsumer.logger$lzycompute(SimpleConsumer.scala:30)
at kafka.consumer.SimpleConsumer.logger(SimpleConsumer.scala:30)
at kafka.utils.Logging$class.info(Logging.scala:67)
at kafka.consumer.SimpleConsumer.info(SimpleConsumer.scala:30)
at kafka.consumer.SimpleConsumer.liftedTree1$1(SimpleConsumer.scala:75)
at kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(SimpleConsumer.scala:69)
at kafka.consumer.SimpleConsumer.getOffsetsBefore(SimpleConsumer.scala:128)
at kafka.javaapi.consumer.SimpleConsumer.getOffsetsBefore(SimpleConsumer.scala:79)
at storm.kafka.KafkaUtils.getOffset(KafkaUtils.java:77)
at storm.kafka.KafkaUtils.getOffset(KafkaUtils.java:67)
at storm.kafka.PartitionManager.<init>(PartitionManager.java:83)
at storm.kafka.ZkCoordinator.refresh(ZkCoordinator.java:98)
at storm.kafka.ZkCoordinator.getMyManagedPartitions(ZkCoordinator.java:69)
at storm.kafka.KafkaSpout.nextTuple(KafkaSpout.java:135)
at backtype.storm.daemon.executor$fn__3373$fn__3388$fn__3417.invoke(executor.clj:565)
at backtype.storm.util$async_loop$fn__464.invoke(util.clj:463) at clojure.lang.AFn.run(AFn.java:24)
at java.lang.Thread.run(Thread.java:745)cg
问题出在哪里,如何解决?
What is the problem and how to solve it?
以下是我包含的依赖项:
Here are the dependencies that I include:
<dependencies>
<dependency>
<groupId>org.apache.storm</groupId>
<artifactId>storm-core</artifactId>
<version>0.9.5</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka_2.10</artifactId>
<version>0.8.2-beta</version>
</dependency>
<dependency>
<groupId>org.apache.storm</groupId>
<artifactId>storm-kafka</artifactId>
<version>0.9.5</version>
</dependency>
<dependency>
<groupId>org.codehaus.jackson</groupId>
<artifactId>jackson-mapper-asl</artifactId>
<version>1.9.11</version>
</dependency>
<dependency>
<groupId>org.java-websocket</groupId>
<artifactId>Java-WebSocket</artifactId>
<version>1.3.0</version>
</dependency>
<dependency>
<groupId>org.twitter4j</groupId>
<artifactId>twitter4j-core</artifactId>
<version>[3.0,)</version>
</dependency>
<dependency>
<groupId>org.twitter4j</groupId>
<artifactId>twitter4j-stream</artifactId>
<version>[3.0,)</version>
</dependency>
</dependencies>
推荐答案
问题是,您应该通过以下方式包含Kafka:
The thing is that you should include Kafka in the following way:
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka_2.10</artifactId>
<version>0.8.1.1</version>
<exclusions>
<exclusion>
<groupId>org.apache.zookeeper</groupId>
<artifactId>zookeeper</artifactId>
</exclusion>
<exclusion>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
</exclusion>
</exclusions>
</dependency>
由于以下原因:
请注意,排除了ZooKeeper和log4j依赖性,以防止与Storm的依赖性发生版本冲突.
Note that the ZooKeeper and log4j dependencies are excluded to prevent version conflicts with Storm's dependencies.
这篇关于KafkaSpout为log4j抛出NoClassDefFoundError的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!
查看全文