Spark SQL 因 java.lang.NoClassDefFoundError 失败:org/codehaus/commons/compiler/UncheckedCompileException [英] Spark SQL fails with java.lang.NoClassDefFoundError: org/codehaus/commons/compiler/UncheckedCompileException
本文介绍了Spark SQL 因 java.lang.NoClassDefFoundError 失败:org/codehaus/commons/compiler/UncheckedCompileException的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
在数据帧上调用第一个操作后,在 Java 中运行 Spark SQL (v2.1.0_2.11) 程序立即失败,并出现以下异常:
Running a Spark SQL (v2.1.0_2.11) program in Java immediately fails with the following exception, as soon as the first action is called on a dataframe:
java.lang.ClassNotFoundException: org.codehaus.commons.compiler.UncheckedCompileException
我在 Eclipse 中运行它,在 spark-submit
环境之外.我使用以下 Spark SQL Maven 依赖项:
I ran it in Eclipse, outside of the spark-submit
environment. I use the following Spark SQL Maven dependency:
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_2.11</artifactId>
<version>2.1.0</version>
<scope>provided</scope>
</dependency>
推荐答案
罪魁祸首是库 commons-compiler
.这是冲突:
The culprit is the library commons-compiler
. Here is the conflict:
要解决此问题,请将以下内容添加到您的 pom.xml:
To work around this, add the following to your pom.xml:
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.codehaus.janino</groupId>
<artifactId>commons-compiler</artifactId>
<version>2.7.8</version>
</dependency>
</dependencies>
</dependencyManagement>
这篇关于Spark SQL 因 java.lang.NoClassDefFoundError 失败:org/codehaus/commons/compiler/UncheckedCompileException的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!
查看全文