Spark SQL失败并出现java.lang.NoClassDefFoundError:org/codehaus/commons/compiler/UncheckedCompileException [英] Spark SQL fails with java.lang.NoClassDefFoundError: org/codehaus/commons/compiler/UncheckedCompileException

问题描述

在数据帧上调用第一个操作后，立即在Java中运行Spark SQL(v2.1.0_2.11)程序会立即失败，但出现以下异常:

Running a Spark SQL (v2.1.0_2.11) program in Java immediately fails with the following exception, as soon as the first action is called on a dataframe:

java.lang.ClassNotFoundException: org.codehaus.commons.compiler.UncheckedCompileException

我在spark-submit环境之外的Eclipse中运行了它.我使用以下Spark SQL Maven依赖项:

I ran it in Eclipse, outside of the spark-submit environment. I use the following Spark SQL Maven dependency:

<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-sql_2.11</artifactId>
    <version>2.1.0</version>
    <scope>provided</scope>
</dependency>

推荐答案

罪魁祸首是库commons-compiler.这是冲突:

The culprit is the library commons-compiler. Here is the conflict:

要解决此问题，请将以下内容添加到pom.xml中:

To work around this, add the following to your pom.xml:

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>org.codehaus.janino</groupId>
            <artifactId>commons-compiler</artifactId>
            <version>2.7.8</version>
        </dependency>
    </dependencies>
</dependencyManagement>

这篇关于Spark SQL失败并出现java.lang.NoClassDefFoundError:org/codehaus/commons/compiler/UncheckedCompileException的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！