关于SBT纱错误在使用火花 [英] Error about sbt yarn at using spark

问题描述

喜当我写这篇code

hi when i am writing this code

>sbt

和看到这个结果后，

beyhan@beyhan:~/sparksample$ sbt
Starting sbt: invoke with -help for other options
[info] Set current project to Spark Sample (in build file:/home/beyhan/sparksample/)

和我写这篇code后

>compile

和我得到这个错误

[error] {file:/home/beyhan/sparksample/}default-f390c8/*:update: sbt.ResolveException: unresolved dependency: org.apache.hadoop#hadoop-yarn-common;1.0.4: not found
    [error] unresolved dependency: org.apache.hadoop#hadoop-yarn-client;1.0.4: not found
    [error] unresolved dependency: org.apache.hadoop#hadoop-yarn-api;1.0.4: not found
    [error] download failed: org.eclipse.jetty.orbit#javax.transaction;1.1.1.v201105210645!javax.transaction.orbit
    [error] download failed: org.eclipse.jetty.orbit#javax.servlet;3.0.0.v201112011016!javax.servlet.orbit
    [error] download failed: org.eclipse.jetty.orbit#javax.mail.glassfish;1.4.1.v201005082020!javax.mail.glassfish.orbit
    [error] download failed: org.eclipse.jetty.orbit#javax.activation;1.1.0.v201105071233!javax.activation.orbit
    [error] Total time: 14 s, completed Oct 16, 2015 3:58:48 PM

我sparksample有这个。

My sparksample has this.

beyhan@beyhan:~/sparksample$ ll
total 20
drwxrwxr-x  4 beyhan beyhan 4096 Eki 16 16:02 ./
drwxr-xr-x 57 beyhan beyhan 4096 Eki 16 15:27 ../
drwxrwxr-x  2 beyhan beyhan 4096 Eki 16 16:02 project/
-rw-rw-r--  1 beyhan beyhan  142 Eki 15 18:57 simple.sbt
drwxrwxr-x  3 beyhan beyhan 4096 Eki 15 11:14 src/

另外的src文件有

Also src file has

src>main>scala>SimpleCode.scala

和我simple.sbt文件中像这样

And my simple.sbt file like this

name := "Spark Sample"

version := "1.0"

scalaVersion := "2.11.7"

libraryDependencies += "org.apache.spark" % "spark-core_2.11" % "1.2.0"

我应该怎么办？
我认为，这个错误我的纱线，因为我没有。
谢谢你。

What should i do? I think this error about my yarn because i don't have. Thanks.

推荐答案

的这种依赖性org.apache.hadoop＃Hadoop的纱线客户端; 1.0.4 为你似乎并不喜欢它是因为 build.sbt 。或许有一些错误的缓存文件〜/ .ivy2 〜/ .m2目录，还是因为也许一些项目/ *。SBT 文件引入额外的依赖关系。

This dependency of org.apache.hadoop#hadoop-yarn-client;1.0.4 for you doesn't seem like it is because of build.sbt. Perhaps there is something wrong with your cached files in ~/.ivy2 or ~/.m2, or maybe because of some project/*.sbt files introducing additional dependencies.

这对我来说工作正常，但：

It works fine for me though:

build.sbt

$ cat build.sbt 
name := "Spark Sample"

version := "1.0"

scalaVersion := "2.11.7"

libraryDependencies += "org.apache.spark" %% "spark-core" % "1.2.0"

所有的依赖解决：

All dependencies resolved:


$ SBT编译
获取org.scala-SBT SBT 0.13.9 ...
::检索:: org.scala-SBT＃启动应用程序
    confs：[默认]
    52文物复制，0已检索（17785kB / 791ms）
获取斯卡拉2.10.5（为SBT）...
::检索:: org.scala-SBT＃启动阶
    confs：[默认]
    5文物复制，0已检索（24493kB / 306ms）
[资讯]设置当前项目星火样品（在构建文件：/家庭/ tuxdna的/ tmp / P /）
[资讯] {更新文件：/家庭/ tuxdna的/ tmp / P /} p ...
[资讯]解决的JLine＃的JLine; 2.12.1 ...
[信息]下载https://repo1.maven.org/maven2/org/apache/spark/spark-core_2.11/1.2.0/spark-core_2.11-1.2.0.jar ...
[信息] [成功] org.apache.spark＃火花core_2.11;！1.2.0火花core_2.11.jar（31007ms）
[信息]下载https://repo1.maven.org/maven2/org/apache/spark/spark-network-common_2.11/1.2.0/spark-network-common_2.11-1.2.0.jar ...
[信息] [成功] org.apache.spark＃火花网络common_2.11;！1.2.0火花网络common_2.11.jar（1873ms）
[信息]下载https://repo1.maven.org/maven2/org/apache/spark/spark-network-shuffle_2.11/1.2.0/spark-network-shuffle_2.11-1.2.0.jar ...
[信息] [成功] org.apache.spark＃火花网络shuffle_2.11;！1.2.0火花网络shuffle_2.11.jar（2122ms）
[信息]完成更新。
[成功]总时间：第61号，完成了2015年10月17日上午12点48分49秒


请注意我的安装Scala和SBT版本：

Note my installed Scala and SBT versions:

$ sbt sbt-version
[info] Set current project to Spark Sample (in build file:/home/tuxdna/tmp/p/)
[info] 0.13.9
$ scala -version
Scala code runner version 2.11.2 -- Copyright 2002-2013, LAMP/EPFL

你能尝试使用一个单独的用户这些步骤（或者一个单独的机器上）？

Could you try these steps using a separate user ( or maybe on a separate machine ) ?

这篇关于关于SBT纱错误在使用火花的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！