解决NoClassDefFoundError:org/apache/spark/Logging异常 [英] Resolving NoClassDefFoundError: org/apache/spark/Logging exception

问题描述

我正在尝试hbase-spark连接器.首先，我尝试

I was trying out hbase-spark connector. To start with, I am trying out this code.

我的pom依赖项是:

<dependencies>
    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-core_2.11</artifactId>
        <version>2.0.0</version>
    </dependency>
    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-sql_2.11</artifactId>
        <version>2.0.0</version>
    </dependency>
    <dependency>
        <groupId>org.apache.hbase</groupId>
        <artifactId>hbase-spark</artifactId>
        <version>2.0.0-alpha4</version>
    </dependency>
</dependencies>

我在运行代码时遇到以下异常:

I am getting following exception while running the code:

线程"main"中的异常java.lang.NoClassDefFoundError:org/apache/spark/Logging在java.lang.ClassLoader.defineClass1(本机方法)在java.lang.ClassLoader.defineClass(ClassLoader.java:760)在java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)在java.net.URLClassLoader.defineClass(URLClassLoader.java:467)在java.net.URLClassLoader.access $ 100(URLClassLoader.java:73)在java.net.URLClassLoader $ 1.run(URLClassLoader.java:368)在java.net.URLClassLoader $ 1.run(URLClassLoader.java:362)在java.security.AccessController.doPrivileged(本机方法)在java.net.URLClassLoader.findClass(URLClassLoader.java:361)在java.lang.ClassLoader.loadClass(ClassLoader.java:424)在sun.misc.Launcher $ AppClassLoader.loadClass(Launcher.java:331)在java.lang.ClassLoader.loadClass(ClassLoader.java:357)在org.apache.hadoop.hbase.spark.JavaHBaseContext.(JavaHBaseContext.scala:46)在com.myproj.poc.sparkhbaseneo4j.App.main(App.java:71)引起原因:java.lang.ClassNotFoundException:org.apache.spark.Logging在java.net.URLClassLoader.findClass(URLClassLoader.java:381)在java.lang.ClassLoader.loadClass(ClassLoader.java:424)在sun.misc.Launcher $ AppClassLoader.loadClass(Launcher.java:331)在java.lang.ClassLoader.loadClass(ClassLoader.java:357)...另外14个

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/spark/Logging at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:760) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at java.net.URLClassLoader.defineClass(URLClassLoader.java:467) at java.net.URLClassLoader.access$100(URLClassLoader.java:73) at java.net.URLClassLoader$1.run(URLClassLoader.java:368) at java.net.URLClassLoader$1.run(URLClassLoader.java:362) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:361) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at org.apache.hadoop.hbase.spark.JavaHBaseContext.(JavaHBaseContext.scala:46) at com.myproj.poc.sparkhbaseneo4j.App.main(App.java:71) Caused by: java.lang.ClassNotFoundException: org.apache.spark.Logging at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 14 more

com.myproj.poc.sparkhbaseneo4j.App.main(App.java:71)是我检查了此线程.它说我应该包括所有库的相同版本.之前，我在pom中有2.3.0版本的spark库.但是我意识到 hbase-spark 的最新版本是 2.0.0 .因此，我将所有Spark库的版本降级为 2.0.0 .但是我仍然遇到同样的异常.

I checked this thread. It says that I should include same versions of all libraries. Earlier, I had 2.3.0 versions of spark libraries in my pom. But I realized that hbase-spark has latest version 2.0.0. So I downgraded versions of of all spark libraries to 2.0.0. But I am still getting the same exception.

或者我是否必须仅坚持使用 1.XX 版本，如此a>回答说它在1.5.2版之后被删除了吗?

Or do I have to stick to 1.X.X versions only for using this, as this answer says its been removed after version 1.5.2?

推荐答案

Sean Owen 在

因此，您应该使用以下版本或较高版本

So you should be using the following or higher version

<!-- https://mvnrepository.com/artifact/commons-logging/commons-logging -->
<dependency>
    <groupId>commons-logging</groupId>
    <artifactId>commons-logging</artifactId>
    <version>1.1.1</version>
</dependency>

已更新

我指定了，现在我收到了NoClassDefFoundError:org/apache/spark/streaming/dstream/DStream

I specified that and now I am getting NoClassDefFoundError: org/apache/spark/streaming/dstream/DStream

对于上述问题，您需要遵循以下依赖性

for the above issue you need following dependency

<!-- https://mvnrepository.com/artifact/org.apache.spark/spark-streaming -->
<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-streaming_2.11</artifactId>
    <version>2.0.0</version>
    <scope>provided</scope>
</dependency>

这篇关于解决NoClassDefFoundError:org/apache/spark/Logging异常的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！