在storm-1.0.0 和pyleus-0.3.0 上运行拓扑时出现IRichBolt 错误 [英] IRichBolt Error when running topology on storm-1.0.0 and pyleus-0.3.0

问题描述

我正在使用 storm-1.0.0、pyleus-0.3.0、centos-6.6 和得到错误

<块引用>

线程main"中的异常java.lang.NoClassDefFoundError:回型/风暴/拓扑/IRichBolt

运行:java -client -Ddaemon.name= -Dstorm.options= -Dstorm.home=/usr/local/apache-storm-1.0.0-Dstorm.log.dir=/usr/local/apache-storm-1.0.0/logs -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp/usr/local/apache-storm-1.0.0/lib/asm-5.0.3.jar:/usr/local/apache-storm-1.0.0/lib/clojure-1.7.0.jar:/usr/local/apache-storm-1.0.0/lib/disruptor-3.3.2.jar:/usr/local/apache-storm-1.0.0/lib/kryo-3.0.3.jar:/usr/local/apache-storm-1.0.0/lib/log4j-api-2.1.jar:/usr/local/apache-storm-1.0.0/lib/log4j-core-2.1.jar:/usr/local/apache-storm-1.0.0/lib/log4j-over-slf4j-1.6.6.jar:/usr/local/apache-storm-1.0.0/lib/log4j-slf4j-impl-2.1.jar:/usr/local/apache-storm-1.0.0/lib/minlog-1.3.0.jar:/usr/local/apache-storm-1.0.0/lib/objenesis-2.1.jar:/usr/local/apache-storm-1.0.0/lib/reflectasm-1.10.1.jar:/usr/local/apache-storm-1.0.0/lib/servlet-api-2.5.jar:/usr/local/apache-storm-1.0.0/lib/slf4j-api-1.7.7.jar:/usr/local/apache-storm-1.0.0/lib/storm-core-1.0.0.jar:/usr/local/apache-storm-1.0.0/lib/storm-rename-hack-1.0.0.jar:xyzTopology.jar:/usr/local/storm/conf:/usr/local/apache-storm-1.0.0/bin-Dstorm.jar=xyz_topology.jar com.yelp.pyleus.PyleusTopologyBuilder --local错误:发生 JNI 错误，请检查您的安装并重试线程main"中的异常 java.lang.NoClassDefFoundError: backtype/storm/topology/IRichBolt在 java.lang.Class.getDeclaredMethods0(Native Method)在 java.lang.Class.privateGetDeclaredMethods(Class.java:2701)在 java.lang.Class.privateGetMethodRecursive(Class.java:3048)在 java.lang.Class.getMethod0(Class.java:3018)在 java.lang.Class.getMethod(Class.java:1784)在 sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)在 sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)引起:java.lang.ClassNotFoundException:backtype.storm.topology.IRichBolt在 java.net.URLClassLoader.findClass(URLClassLoader.java:381)在 java.lang.ClassLoader.loadClass(ClassLoader.java:424)在 sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)在 java.lang.ClassLoader.loadClass(ClassLoader.java:357)……还有 7 个pyleus 本地:错误:[StormError] Storm 命令失败.使用 --verbose 运行以获取更多信息.

在使用旧版本的 storm-0.10.0 和 pyleus-0.3.0 时，相同的拓扑运行良好且没有任何错误.

我查看了storm-0.10.0 和storm-1.0.0 文档，发现在storm-1.0.0

(a) 包结构已经改变，

(b) IRichBolt 类没有出现，取而代之的是两个新类——BaseStatefulBoltExecutor 和 BasicBoltExecutor——它们实现了 IRichBolt 接口.

似乎新的 storm-1.0.0 版本无法与 pyleus-0.3.0 一起使用，因为它与旧的 storm-0.10.0 版本.

解决问题并允许在 storm-1.0.0 和 pyleus-0.3.0 上运行拓扑的最佳解决方案是什么em> ?

解决方案

Storm 1.0.0 对包名称进行了重大重构.所有包 backtype/* 都被 org/apache/storm 替换了.因此，您需要使用 Storm 1.0.0 重新编译您的拓扑(包括一些代码更改；即，使用不同的 import 来对齐新的包结构).

作为替代方案，您可以在 Storm 集群中启用向后兼容性(请参阅 https://storm.apache.org/releases/1.0.0/index.html)

<块引用>

注意

在最新版本中，类包已经从backtype.storm"更改为org.apache.storm"，因此旧版本编译的拓扑代码不会像这样在Storm 1.0.0上运行.可通过以下配置实现向后兼容性

client.jartransformer.class: "org.apache.storm.hack.StormShadeTransformer"

如果要运行用旧版本storm编译的代码，需要在storm安装中添加以上配置.配置应添加到您用于提交拓扑的机器中.

参考https://issues.apache.org/jira/browse/STORM-1202 了解更多详情.

I'm running storm topology "pyleus --verbose local xyz_topology.jar" using storm-1.0.0, pyleus-0.3.0, centos-6.6 and getting the Error

Exception in thread "main" java.lang.NoClassDefFoundError: backtype/storm/topology/IRichBolt

Running: java -client -Ddaemon.name= -Dstorm.options= -Dstorm.home=/usr/local/apache-storm-1.0.0
-Dstorm.log.dir=/usr/local/apache-storm-1.0.0/logs -Djava.library.path=/usr/local/    
lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /usr/local/apache-storm-1.0.0/lib/asm-5.0.3.jar:
/usr/local/apache-storm-1.0.0/lib/clojure-1.7.0.jar:/usr/local/apache-storm-1.0.0/lib/disruptor-3.3.2.jar:
/usr/local/apache-storm-1.0.0/lib/kryo-3.0.3.jar:
/usr/local/apache-storm-1.0.0/lib/log4j-api-2.1.jar:/usr/local/apache-storm-1.0.0/lib/log4j-core-2.1.jar    
:/usr/local/apache-storm-1.0.0/lib/log4j-over-slf4j-1.6.6.jar:
/usr/local/apache-storm-1.0.0/lib/log4j-slf4j-impl-2.1.jar:/usr/local/apache-storm-1.0.0/lib/minlog-1.3.0.jar:
/usr/local/apache-storm-1.0.0/lib/objenesis-2.1.jar:/usr/local/apache-storm-1.0.0/lib/reflectasm-1.10.1.jar:
/usr/local/apache-storm-1.0.0/lib/servlet-api-2.5.jar:/usr/local/apache-storm-1.0.0/lib/slf4j-api-1.7.7.jar:
/usr/local/apache-storm-1.0.0/lib/storm-core-1.0.0.jar:
/usr/local/apache-storm-1.0.0/lib/storm-rename-hack-1.0.0.jar:xyzTopology.jar:
/usr/local/storm/conf:/usr/local/apache-storm-1.0.0/bin
-Dstorm.jar=xyz_topology.jar com.yelp.pyleus.PyleusTopologyBuilder --local
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.NoClassDefFoundError: backtype/storm/topology/IRichBolt
        at java.lang.Class.getDeclaredMethods0(Native Method)
        at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
        at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
        at java.lang.Class.getMethod0(Class.java:3018)
        at java.lang.Class.getMethod(Class.java:1784)
        at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
        at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)
Caused by: java.lang.ClassNotFoundException: backtype.storm.topology.IRichBolt
        at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        ... 7 more
pyleus local: error: [StormError] Storm command failed. Run with --verbose for more info.

The same topology worked well and without any errors when using the old version of storm-0.10.0 and pyleus-0.3.0.

I looked at the storm-0.10.0 and storm-1.0.0 docs and found that in storm-1.0.0

(a) the package structure has been changed and

(b) the IRichBolt class is not presented and instead of that there are two new classes - BaseStatefulBoltExecutor and BasicBoltExecutor - which implement IRichBolt interface.

It seems new storm-1.0.0 release can't work with pyleus-0.3.0 as it was with old storm-0.10.0 version.

What would be the best solution to resolve the issue and allow to run the topology on storm-1.0.0 and pyleus-0.3.0 ?

解决方案

Storm 1.0.0 includes a major refactoring with regard to package names. All packages backtype/* got replaced by org/apache/storm. Thus, you need to recompile your topology using Storm 1.0.0 (including some code change; ie, using different imports to align to the new package structure).

As an alternative, you can enable backward compatibility in your Storm cluster (see https://storm.apache.org/releases/1.0.0/index.html)

NOTE

In the latest version, the class packages have been changed from "backtype.storm" to "org.apache.storm" so the topology code compiled with older version won't run on the Storm 1.0.0 just like that. Backward compatibility is available through following configuration

client.jartransformer.class: "org.apache.storm.hack.StormShadeTransformer"

You need to add the above config in storm installation if you want to run the code compiled with older versions of storm. The config should be added in the machine you use to submit your topologies.

Refer to https://issues.apache.org/jira/browse/STORM-1202 for more details.

这篇关于在storm-1.0.0 和pyleus-0.3.0 上运行拓扑时出现IRichBolt 错误的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！