星火提交尽管它在jar文件中存在没有找到主类 [英] Spark-submit not finding main class although it does exist in the jar file

问题描述

第一：这里是提交申请的jar文件：

  $ LS -rlta /shared/ysgood/target/yardstick-spark-uber-0.0.1.jar
-rw-R  -  R-- 1史蒂夫员工138611565 8月6日01:41 /shared/ysgood/target/yardstick-spark-uber-0.0.1.jar
 

下面是要提交的类：

  01：55：02 / ysgood $罐子-tvf目标/准绳火花 - 尤伯杯0.0.1.jar | grep的SparkCoreRDDBenchmark.class
 15091星期四8月6日1点36分三十零秒PDT 2015组织/准绳/火花/ SparkCoreRDDBenchmark.class
 

下面是企图在提交：

  $火花提交--master $ MASTER --class org.yardstick.spark.SparkCoreRDDBenchmark目标/准绳火花 - 尤伯杯0.0.1.jar注：SPARK_ prePEND_CLASSES设置，将本地编译星火类提前组装。
错误：无法加载从JAR主类
文件：/shared/ysgood/org.yardstick.spark.SparkCoreRDDBenchmark
 

关于错误：注意路径罐子不正确：以下

  

/shared/ysgood/org.yardstick.spark.SparkCoreRDDBenchmark

没有意义：它缺少路径jar文件

  

目标/准绳火花 - 尤伯杯0.0.1.jar

解决方案

你必须把你的主类的包没有路径，主类级后。 c中的SparkCoreRDDBenchmark类是在包org.yardstick.spark所以，检查是否在$ C $。
如果是，则尽量不星火运行你的罐子，看是否得到错误无法加载主类。也许那里有一些问题，当罐子被创建。
祝你好运！

First: here is the application jar file to be submitted:

$ls -rlta /shared/ysgood/target/yardstick-spark-uber-0.0.1.jar
-rw-r--r--  1 steve  staff  138611565 Aug  6 01:41 /shared/ysgood/target/yardstick-spark-uber-0.0.1.jar

Here is the class to be submitted:

01:55:02/ysgood $jar -tvf target/yardstick-spark-uber-0.0.1.jar | grep SparkCoreRDDBenchmark.class
 15091 Thu Aug 06 01:36:30 PDT 2015 org/yardstick/spark/SparkCoreRDDBenchmark.class

Here is the attempt at submitting:

$spark-submit --master $MASTER --class org.yardstick.spark.SparkCoreRDDBenchmark target/yardstick-spark-uber-0.0.1.jar 

NOTE: SPARK_PREPEND_CLASSES is set, placing locally compiled Spark classes ahead of assembly.
Error: Cannot load main class from JAR 
file:/shared/ysgood/org.yardstick.spark.SparkCoreRDDBenchmark

Regarding the error: notice the path to the jar is incorrect: the following

/shared/ysgood/org.yardstick.spark.SparkCoreRDDBenchmark

does not make sense: it is missing the path to the jar file

target/yardstick-spark-uber-0.0.1.jar

解决方案

After -class you have to put the package of your main class not the path to the main class. So check if in your code the SparkCoreRDDBenchmark class is in the package org.yardstick.spark. If it is, then try running your jar without Spark, see if you get the error Cannot load main class. Maybe there where some problems, when the jar was created. Good luck!

这篇关于星火提交尽管它在jar文件中存在没有找到主类的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！