将jsoup添加为提供的依赖项后，Storm命令失败，并显示NoClassDefFoundError [英] Storm command fails with NoClassDefFoundError after adding jsoup as provided dependency
问题描述
我在项目中使用JSoup，并且在我的POM文件中声明了依赖项.它可以很好地编译并且也可以很好地运行，但是仅当我使用jar with all dependencies并将其依赖范围更改为compiled时才可以.
I'm using JSoup in my project and I've declared the dependency in my POM file. It compiles just fine and runs fine too, but only when I used the jar with all dependencies and change the have the scope of the dependency to compiled.
如果我将此范围更改为provided，那么我仍然可以编译，但不能运行它.它给了我ClassNotFoundException.我在classpath和path变量中都包含了必要的JAR文件，但是我仍然面临着这个问题.
If I change this scope to provided, then I can still compile just fine, but not run it. It gives me the ClassNotFoundException. I have included the necessary JAR file in the classpath and also the path variables but I'm still facing this problem.
我可以使用compile选项，但是我真的很讨厌为什么我不能使用提供的选项来运行它，如果有人使用，我将真的表示感谢可以帮助我弄清楚为什么.
I can get working with the compile option but it's really irking me at the back of my mind why I can't get it running with the provided option, and I would really appreciate it if someone could help me figure why.
以下是我看到的错误:
java.lang.NoClassDefFoundError: Lorg/jsoup/nodes/Document;
at java.lang.Class.getDeclaredFields0(Native Method)
at java.lang.Class.privateGetDeclaredFields(Class.java:2300)
at java.lang.Class.getDeclaredField(Class.java:1882)
at java.io.ObjectStreamClass.getDeclaredSUID(ObjectStreamClass.java:1605)
at java.io.ObjectStreamClass.access$700(ObjectStreamClass.java:50)
at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:423)
at java.security.AccessController.doPrivileged(Native Method)
at java.io.ObjectStreamClass.<init>(ObjectStreamClass.java:411)
at java.io.ObjectStreamClass.lookup(ObjectStreamClass.java:308)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1114)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:330)
at backtype.storm.utils.Utils.serialize(Utils.java:52)
at backtype.storm.topology.TopologyBuilder.createTopology(TopologyBuilder.java:94)
at com.yahoo.amit.wordstorm.WordStormTopology.main(WordStormTopology.java:25)
Caused by: java.lang.ClassNotFoundException: org.jsoup.nodes.Document
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
... 14 more
以下是我的POM文件:
Following is my POM file:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.yahoo.amit.wordstorm</groupId>
<artifactId>wordstorm</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>jar</packaging>
<name>wordstorm</name>
<url>http://maven.apache.org</url>
<repositories>
<repository>
<id>clojars.org</id>
<url>http://clojars.org/repo</url>
</repository>
</repositories>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencies>
<dependency>
<groupId>storm</groupId>
<artifactId>storm</artifactId>
<version>0.8.2</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.jsoup</groupId>
<artifactId>jsoup</artifactId>
<version>1.7.2</version>
<scope>provided</scope>
</dependency>
</dependencies>
<build>
<plugins>
<!--
bind the maven-assembly-plugin to the package phase
this will create a jar file without the storm dependencies
suitable for deployment to a cluster.
-->
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
<archive>
<manifest>
<mainClass></mainClass>
</manifest>
</archive>
</configuration>
<executions>
<execution>
<id>make-assembly</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
以下是我的系统变量:
echo $PATH
/Users/programmerman/Summer
Project/apache-maven-3.0.5/bin/:/Users/programmerman/Summer
Project/storm-0.8.2/bin/:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/Users/programmerman/Summer
Project/CLASSPATH/jsoup-1.7.2.jar:/Users/programmerman/Summer
Project/CLASSPATH/*
echo $CLASSPATH
/Users/programmerman/Summer
Project/storm-0.8.2/storm-0.8.2.jar:/Users/programmerman/Summer
Project/storm-0.8.2/lib/*:/Users/programmerman/Summer
Project/storm-0.8.2/conf/storm.yaml:/Users/programmerman/SummerProject/storm-starter-masterPOM/target/storm-starter-0.0.1-SNAPSHOT-jar-with-dependencies.jar:/Users/programmerman/Summer
Project/CLASSPATH/jsoup-1.7.2.jar:/Users/programmerman/Summer
Project/CLASSPATH/*
推荐答案
这既是关于Maven的问题，也是关于Storm及其部署模型的问题.您必须查看storm命令的实际作用.首先，它实际上是一个Python脚本，最终会调用java.
This is as much a question about Maven as it is about Storm and its deployment model. You have to check out what the storm command actually does. First of all, it's actually a Python script that ultimately calls java.
如果查看函数get_classpath(extrajars)，您会注意到它根本不使用$CLASSPATH evironment变量.相反，它将加载核心Storm jar和相对于工作目录的目录lib/中的所有jar，以及~/.storm
If you look at the function get_classpath(extrajars), you'll note that it does not use the $CLASSPATH evironment variable at all. Rather, it loads the core Storm jars and any jars that you have under a directory lib/ relative to your working directory, as well as config files under ~/.storm
(您会发现忽略$CLASSPATH在许多Java应用程序中非常常见.通常，启动脚本"要做的第一件事就是覆盖CLASSPATH或根本不使用它.这是为了防止未知/不受支持的/更早的版本.版本的jar会导致应用程序出现问题).
(You will find that ignoring $CLASSPATH is very common in many Java applications. Usually the first thing a "launch script" does is overwrite the CLASSPATH or not use it at all. This is to prevent unknown / unsupported / earlier versions of your jars from causing problems in your application).
关于当jsoup被声明为提供"时，您的应用程序失败:当您将jar声明为提供的依赖项时，它将不会打包在您的带有依赖项的jar"程序集中.请参阅以下问题以获取良好的解释: Maven范围之间的差异编译并提供JAR包装
As to your application fails when jsoup is declared "provided": when you declare the jar as a provided dependency, it will not be packaged in your "jar with dependencies" assembly. See this question for a good explanation: Difference between maven scope compile and provided for JAR packaging
tl; dr的解释是，编译范围是随uber-jar一起提供的，但前提是没有，因为预期它是由部署到的容器提供"的.通常，容器"是Java Web服务器，例如Tomcat(因此，您永远不必随Java Web应用程序一起交付JSP或Servlet jar).在这种情况下，您希望提供"类的容器"是Storm.但是，Storm没有提供jsoup，因此会出错.
The tl;dr explanation is that compile scope is shipped with your uber-jar, provided scope isn't, because it's expected to be "provided" by the container you are deploying to. Typically, the "container" is a Java web server, like Tomcat (hence, you should never have to ship JSP or Servlet jars with your web apps in Java). In this case, the "container" that you are expecting to "provide" classes is Storm. However, jsoup is not provided by Storm, hence your error.
编译范围类仍需要随应用程序一起提供，因为您的应用程序将实例化/使用接口，枚举等.
Compile-scope classes still need to be shipped with your application because your application will be instantiating / using interfaces, enums, etc.
我的建议是只声明jsoup编译"作用域并继续.替代方法是编写您自己的定制部署脚本和/或程序集，将jsoup置于lib/下-最终本质上是相同的.
My recommendation is to just declare jsoup "compile" scope and move on. The alternative will be to write your own bespoke deployment script and/or assembly that puts jsoup under lib/ - essentially the same thing in the end.
这篇关于将jsoup添加为提供的依赖项后，Storm命令失败，并显示NoClassDefFoundError的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
