Hadoop NoSuchMethodError apache.commons.cli [英] Hadoop NoSuchMethodError apache.commons.cli
问题描述
我正在使用 hadoop-2.7.2
,并用IntelliJ做了一个MapReduceJob。在我的工作中,我使用 apache.commons.cli-1.3.1
,并将该lib放入jar中。
当我在我的Hadoop集群上使用MapReduceJob时,我有一个 NoSuchMethodError
:
线程main中的异常java.lang.NoSuchMethodError:
org.apache.commons.cli.Option.builder(Ljava / lang / String;)Lorg / apache / commons / cli / Option $ Builder;
我不明白,因为该方法存在于类选项
和类选项
从 commons-cli.jar
中提取我的应用程序jar。此外,我没有与我的其他图书馆有这个问题。
谢谢你的时间。
这个问题似乎与如何加载 classloader
类。由于静态生成器类
在 common-cli 1.4
中,而一些 hadoop
依赖关系仍然指旧版本 - 发生问题。
在我的情况下,通过更改jar文件添加顺序来解决问题shell脚本中的classpath负责在程序执行前设置环境。之前,我将jar添加到类路径中,如
CLASSPATH =< Hadoop Jars>:< Common CLI jar>:$ CLASSPATH
更改为
CLASSPATH =< Common CLI jar>:< Hadoop Jars>:$ CLASSPATH
它解决了问题。
I'm using hadoop-2.7.2
and I did a MapReduceJob with IntelliJ. In my job, I'm using apache.commons.cli-1.3.1
and I put the lib in the jar.
When I use the MapReduceJob on my Hadoop cluster I have a NoSuchMethodError
:
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.commons.cli.Option.builder(Ljava/lang/String;)Lorg/apache/commons/cli/Option$Builder;
I don't understand because the method exist in the class Option
and the class Option
is extracted from the commons-cli.jar
to my application jar. Moreover, I don't have this issue with my others libraries.
Thank you for your time.
The problem seems to be related to how the classloader
is loading the classes. Because the static Builder class
was in common-cli 1.4
, while some of the hadoop
dependencies were still referring to older version - the issue occurred.
In my case, the issue resolved by changing the sequence of jar file addition into the classpath in the shell script responsible for setting up environment before program execution. Earlier, I was adding the jar into classpath like
CLASSPATH=<Hadoop Jars>:<Common CLI jar>:$CLASSPATH
is changed to
CLASSPATH=<Common CLI jar>:<Hadoop Jars>:$CLASSPATH
and it fixed the problem.
这篇关于Hadoop NoSuchMethodError apache.commons.cli的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!