使用Spark和IntelliJ时出现NoSuchMethodError [英] NoSuchMethodError when using Spark and IntelliJ
本文介绍了使用Spark和IntelliJ时出现NoSuchMethodError的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
我是Scala和Spark的新手.使事情与IntelliJ一起工作有多么艰辛,我感到沮丧.目前,我无法运行下面的代码.我敢肯定这很简单,但是我无法使其正常工作.
I'm new to Scala and Spark. I've been frustrated by how hard it has been to get things to work with IntelliJ. Currently, I can't get run the code below. I'm sure it's something simple, but I can't get it to work.
我正在尝试运行:
import org.apache.spark.{SparkConf, SparkContext}
object TestScala {
def main(args: Array[String]): Unit = {
val conf = new SparkConf()
conf.setAppName("Datasets Test")
conf.setMaster("local[2]")
val sc = new SparkContext(conf)
println(sc)
}
}
我得到的错误是:
Exception in thread "main" java.lang.NoSuchMethodError: scala.Predef$.refArrayOps([Ljava/lang/Object;)Lscala/collection/mutable/ArrayOps;
at org.apache.spark.util.Utils$.getCallSite(Utils.scala:1413)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:77)
at TestScala$.main(TestScala.scala:13)
at TestScala.main(TestScala.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)
我的build.sbt文件:
My build.sbt file:
name := "sparkBook"
version := "1.0"
scalaVersion := "2.12.1"
推荐答案
将scalaVersion
更改为2.11.8
并将Spark依赖项添加到build.sbt
:
Change your scalaVersion
to 2.11.8
and add the Spark dependency to your build.sbt
:
libraryDependencies += "org.apache.spark" % "spark-core_2.11" % "2.0.2"
这篇关于使用Spark和IntelliJ时出现NoSuchMethodError的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!
查看全文