Storm 导致对 Ignite log4j 的依赖冲突 [英] Storm causes dependency conflicts on Ignite log4j

查看：18
本文介绍了Storm 导致对 Ignite log4j 的依赖冲突的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！

问题描述

我尝试在 Storm 集群上运行一个 Storm 拓扑拓扑 jar 是由提供的创建的.在拓扑螺栓内部，我想从 MyIgniteCache 模块读取数据，但出现以下错误.我认为依赖项(:/usr/hdp/2.6.0.3-8/storm/lib/log4j-slf4j-impl-2.8.jar:/usr/hdp/2.6.0.3-8/storm/lib/log4j-core-Storm-core 的 2.8.jar:log4j) 导致 ignite-log4j 冲突.

I try to run a storm topology on a Storm cluster The topology jar is created by provided. Inside topology bolt i want to read data from MyIgniteCache module but i get such a following error. I think the dependencies (:/usr/hdp/2.6.0.3-8/storm/lib/log4j-slf4j-impl-2.8.jar:/usr/hdp/2.6.0.3-8/storm/lib/log4j-core-2.8.jar:log4j) of storm-core cause conflict over ignite-log4j.

ava.lang.IncompatibleClassChangeError:实现类在 java.lang.ClassLoader.defineClass1(Native Method) ~[?:1.8.0_112]在 java.lang.ClassLoader.defineClass(ClassLoader.java:763) ~[?:1.8.0_112]在 java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) ~[?:1.8.0_112]在 java.net.URLClassLoader.defineClass(URLClassLoader.java:467) ~[?:1.8.0_112]在 java.net.URLClassLoader.access$100(URLClassLoader.java:73) ~[?:1.8.0_112]在 java.net.URLClassLoader$1.run(URLClassLoader.java:368) ~[?:1.8.0_112]在 java.net.URLClassLoader$1.run(URLClassLoader.java:362) ~[?:1.8.0_112]在 java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_112]在 java.net.URLClassLoader.findClass(URLClassLoader.java:361) ~[?:1.8.0_112]在 java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_112]在 sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) ~[?:1.8.0_112]在 java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_112]在 java.lang.Class.forName0(Native Method) ~[?:1.8.0_112]在 java.lang.Class.forName(Class.java:264) ~[?:1.8.0_112]在 org.apache.ignite.internal.util.IgniteUtils.addLog4jNoOpLogger(IgniteUtils.java:8366) ~[stormjar.jar:?]在 org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:930) ~[stormjar.jar:?]在 org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:854) ~[stormjar.jar:?]在 org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:724) ~[stormjar.jar:?]在 org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:693) ~[stormjar.jar:?]在 org.apache.ignite.Ignition.start(Ignition.java:352) ~[stormjar.jar:?]

ava.lang.IncompatibleClassChangeError: Implementing class at java.lang.ClassLoader.defineClass1(Native Method) ~[?:1.8.0_112] at java.lang.ClassLoader.defineClass(ClassLoader.java:763) ~[?:1.8.0_112] at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) ~[?:1.8.0_112] at java.net.URLClassLoader.defineClass(URLClassLoader.java:467) ~[?:1.8.0_112] at java.net.URLClassLoader.access$100(URLClassLoader.java:73) ~[?:1.8.0_112] at java.net.URLClassLoader$1.run(URLClassLoader.java:368) ~[?:1.8.0_112] at java.net.URLClassLoader$1.run(URLClassLoader.java:362) ~[?:1.8.0_112] at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_112] at java.net.URLClassLoader.findClass(URLClassLoader.java:361) ~[?:1.8.0_112] at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_112] at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) ~[?:1.8.0_112] at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_112] at java.lang.Class.forName0(Native Method) ~[?:1.8.0_112] at java.lang.Class.forName(Class.java:264) ~[?:1.8.0_112] at org.apache.ignite.internal.util.IgniteUtils.addLog4jNoOpLogger(IgniteUtils.java:8366) ~[stormjar.jar:?] at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:930) ~[stormjar.jar:?] at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:854) ~[stormjar.jar:?] at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:724) ~[stormjar.jar:?] at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:693) ~[stormjar.jar:?] at org.apache.ignite.Ignition.start(Ignition.java:352) ~[stormjar.jar:?]

我的storm拓扑模块的依赖树如下

And the dependency tree of my storm topology module is like below

<p>
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building mytopology-enrichment 1.1
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ mytopology-enrichment---
[INFO] com.my.storm:mytopology-enrichment:jar:1.1
[INFO] +- org.apache.storm:flux-core:jar:1.1.0:compile
[INFO] |  \- commons-cli:commons-cli:jar:1.2:compile
[INFO] +- org.apache.storm:flux-wrappers:jar:1.1.0:compile
[INFO] |  +- org.apache.storm:multilang-javascript:jar:1.1.0:compile
[INFO] |  +- org.apache.storm:multilang-ruby:jar:1.1.0:compile
[INFO] |  \- org.apache.storm:multilang-python:jar:1.1.0:compile
[INFO] +- com.my.storm:my-common:jar:1.1:compile
[INFO] +- commons-logging:commons-logging:jar:1.1.3:compile
[INFO] +- com.my.storm:mytopology-preprocessing:jar:1.1:compile
[INFO] |  \- org.apache.commons:commons-lang3:jar:3.7:compile
[INFO] +- org.apache.storm:storm-core:jar:1.1.0:compile
[INFO] |  +- com.esotericsoftware:kryo:jar:3.0.3:compile
[INFO] |  |  +- com.esotericsoftware:reflectasm:jar:1.10.1:compile
[INFO] |  |  |  \- org.ow2.asm:asm:jar:5.0.3:compile
[INFO] |  |  +- com.esotericsoftware:minlog:jar:1.3.0:compile
[INFO] |  |  \- org.objenesis:objenesis:jar:2.1:compile
[INFO] |  +- org.clojure:clojure:jar:1.7.0:compile
[INFO] |  +- ring-cors:ring-cors:jar:0.1.5:compile
[INFO] |  +- com.lmax:disruptor:jar:3.3.2:compile
[INFO] |  +- org.apache.logging.log4j:log4j-api:jar:2.8:compile
[INFO] |  +- org.apache.logging.log4j:log4j-core:jar:2.8:compile
[INFO] |  +- org.apache.logging.log4j:log4j-slf4j-impl:jar:2.8:compile
[INFO] |  +- org.slf4j:log4j-over-slf4j:jar:1.6.6:compile
[INFO] |  +- javax.servlet:servlet-api:jar:2.5:compile
[INFO] |  \- org.slf4j:slf4j-api:jar:1.7.21:compile
[INFO] +- org.apache.storm:storm-kafka-client:jar:1.1.0:compile
[INFO] +- org.apache.kafka:kafka_2.10:jar:0.10.2.0:compile
[INFO] |  +- net.sf.jopt-simple:jopt-simple:jar:5.0.3:compile
[INFO] |  +- com.yammer.metrics:metrics-core:jar:2.2.0:compile
[INFO] |  +- org.scala-lang:scala-library:jar:2.10.6:compile
[INFO] |  \- com.101tec:zkclient:jar:0.10:compile
[INFO] +- org.apache.kafka:kafka-clients:jar:0.10.2.0:compile
[INFO] |  +- net.jpountz.lz4:lz4:jar:1.3.0:compile
[INFO] |  \- org.xerial.snappy:snappy-java:jar:1.1.2.6:compile
[INFO] +- junit:junit:jar:4.12:compile
[INFO] |  \- org.hamcrest:hamcrest-core:jar:1.3:compile
[INFO] +- joda-time:joda-time:jar:2.10:compile
[INFO] +- com.googlecode.json-simple:json-simple:jar:1.1.1:compile
[INFO] +- com.my.ignite:my-config-cache:jar:1.1:compile
[INFO] |  +- org.apache.ignite:ignite-core:jar:2.5.0:compile
[INFO] |  |  +- javax.cache:cache-api:jar:1.0.0:compile
[INFO] |  |  +- org.jetbrains:annotations:jar:13.0:compile
[INFO] |  |  \- org.gridgain:ignite-shmem:jar:1.0.0:compile
[INFO] |  +- org.apache.ignite:ignite-spring:jar:2.5.0:compile
[INFO] |  |  +- org.apache.ignite:ignite-indexing:jar:2.5.0:compile
[INFO] |  |  |  +- org.apache.lucene:lucene-core:jar:5.5.2:compile
[INFO] |  |  |  +- org.apache.lucene:lucene-analyzers-common:jar:5.5.2:compile
[INFO] |  |  |  +- org.apache.lucene:lucene-queryparser:jar:5.5.2:compile
[INFO] |  |  |  |  +- org.apache.lucene:lucene-queries:jar:5.5.2:compile
[INFO] |  |  |  |  \- org.apache.lucene:lucene-sandbox:jar:5.5.2:compile
[INFO] |  |  |  \- com.h2database:h2:jar:1.4.195:compile
[INFO] |  |  +- org.springframework:spring-core:jar:4.3.7.RELEASE:compile
[INFO] |  |  +- org.springframework:spring-aop:jar:4.3.7.RELEASE:compile
[INFO] |  |  +- org.springframework:spring-beans:jar:4.3.7.RELEASE:compile
[INFO] |  |  +- org.springframework:spring-context:jar:4.3.7.RELEASE:compile
[INFO] |  |  +- org.springframework:spring-expression:jar:4.3.7.RELEASE:compile
[INFO] |  |  +- org.springframework:spring-tx:jar:4.3.7.RELEASE:compile
[INFO] |  |  \- org.springframework:spring-jdbc:jar:4.3.7.RELEASE:compile
[INFO] |  +- org.apache.ignite:ignite-log4j:jar:2.5.0:compile
[INFO] |  |  \- log4j:log4j:jar:1.2.17:compile
[INFO] |  \- postgresql:postgresql:jar:9.1-901.jdbc4:compile
[INFO] +- com.my.elastic:my-elasticsearch:jar:1.1:compile
[INFO] |  +- org.elasticsearch:elasticsearch-storm:jar:5.4.1:compile
[INFO] |  +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile
[INFO] |  +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile
[INFO] |  \- commons-httpclient:commons-httpclient:jar:3.1:compile
[INFO] |     \- commons-codec:commons-codec:jar:1.2:compile
[INFO] \- com.google.code.gson:gson:jar:2.8.0:compile
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2.351 s
[INFO] Finished at: 2018-08-14T13:37:44+03:00
[INFO] Final Memory: 16M/294M
</p>

这是我的配置缓存的依赖树

And here is dependency tree of my config cache 

apache ignite cache module dependency tree
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building my-config-cache 1.1
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ my-config-cache ---
[INFO] com.my.ignite:my-config-cache:jar:1.1
[INFO] +- org.apache.ignite:ignite-core:jar:2.5.0:compile
[INFO] |  +- javax.cache:cache-api:jar:1.0.0:compile
[INFO] |  +- org.jetbrains:annotations:jar:13.0:compile
[INFO] |  \- org.gridgain:ignite-shmem:jar:1.0.0:compile
[INFO] +- org.apache.ignite:ignite-spring:jar:2.5.0:compile
[INFO] |  +- org.apache.ignite:ignite-indexing:jar:2.5.0:compile
[INFO] |  |  +- commons-codec:commons-codec:jar:1.11:compile
[INFO] |  |  +- org.apache.lucene:lucene-core:jar:5.5.2:compile
[INFO] |  |  +- org.apache.lucene:lucene-analyzers-common:jar:5.5.2:compile
[INFO] |  |  +- org.apache.lucene:lucene-queryparser:jar:5.5.2:compile
[INFO] |  |  |  +- org.apache.lucene:lucene-queries:jar:5.5.2:compile
[INFO] |  |  |  \- org.apache.lucene:lucene-sandbox:jar:5.5.2:compile
[INFO] |  |  \- com.h2database:h2:jar:1.4.195:compile
[INFO] |  +- org.springframework:spring-core:jar:4.3.7.RELEASE:compile
[INFO] |  +- org.springframework:spring-aop:jar:4.3.7.RELEASE:compile
[INFO] |  +- org.springframework:spring-beans:jar:4.3.7.RELEASE:compile
[INFO] |  +- org.springframework:spring-context:jar:4.3.7.RELEASE:compile
[INFO] |  +- org.springframework:spring-expression:jar:4.3.7.RELEASE:compile
[INFO] |  +- org.springframework:spring-tx:jar:4.3.7.RELEASE:compile
[INFO] |  \- org.springframework:spring-jdbc:jar:4.3.7.RELEASE:compile
[INFO] +- org.apache.ignite:ignite-log4j:jar:2.5.0:compile
[INFO] |  \- log4j:log4j:jar:1.2.17:compile
[INFO] +- commons-logging:commons-logging:jar:1.1.3:compile
[INFO] +- com.google.code.gson:gson:jar:2.8.0:compile
[INFO] +- junit:junit:jar:4.12:compile
[INFO] |  \- org.hamcrest:hamcrest-core:jar:1.3:compile
[INFO] +- postgresql:postgresql:jar:9.1-901.jdbc4:compile
[INFO] \- com.googlecode.json-simple:json-simple:jar:1.1.1:compile
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2.070 s
[INFO] Finished at: 2018-08-14T13:40:24+03:00
[INFO] Final Memory:

尽管存在这些依赖关系，但我在 IDE 上运行拓扑和 myconfigcache 模块时收到了另一个严重警告

Although these dependencies, i get another severe warnings while running the topology and myconfigcache module on IDE

Strom topoloji module runtime

log4j:WARN No appenders could be found for logger (org.elasticsearch.storm.EsBolt).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

bla blaa blaa
........
...........


Aug 14, 2018 1:36:17 PM java.util.logging.LogManager$RootLogger log
SEVERE: Failed to resolve default logging config file: config/java.util.logging.properties
[13:36:17]    __________  ________________ 
[13:36:17]   /  _/ ___/ |/ /  _/_  __/ __/ 
[13:36:17]  _/ // (7 7    // /  / / / _/   
[13:36:17] /___/\___/_/|_/___/ /_/ /___/  
[13:36:17] 
[13:36:17] ver. 2.5.0#20180523-sha1:86e110c7

blaaa blaa

那么，除了 Storm log4j 依赖项之外，我如何处理运行我的拓扑?

So how can i handle running my topology excluding Storm log4j dependencies ?

推荐答案

Storm 使用 Log4j 2 进行日志记录，所以如果你想连接到它，你应该使用 ignite-log4j2，ignite-log4j 看起来像是用于 Log4j 1.我还希望您可以使用 ignite-slf4j 并完成这项工作.

Storm uses Log4j 2 for logging, so if you want to hook into that you should use ignite-log4j2 instead, ignite-log4j looks like it's for Log4j 1. I'd also expect that you could use ignite-slf4j and have that work.

这篇关于Storm 导致对 Ignite log4j 的依赖冲突的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！

查看全文
相关文章
其他开发最新文章
热门教程
热门工具
登录 关闭
扫码关注1秒登录
发送“验证码”获取 | 15天全站免登陆