斯卡拉code在SBT不编译 [英] Scala code not compiling in SBT

问题描述

我已经写了一张这是对斯卡拉外壳完美运行的机器学习code的。我在使用SBT编译code和创建一个JAR的过程。我使用了一些codeS由例子（以星火例如LocalLR和SparkPI），以试图编译codeS在一个新的项目文件夹。他们都成功编译，但由于某种原因，我的code未编译。我下面所有的目录中约定，但仍然没有成功。

I have written a piece of Machine learning code which is running perfectly on the Scala shell. I am in the process of compiling the code using SBT and creating a JAR. I used some codes from examples(in Spark e.g LocalLR and SparkPI) to try to compile the codes in a new project folder. They all compiled successfully but for some reasons my code is not compiling. I am following all the directory conventions but still no success.

    import org.apache.spark.SparkContext
    import org.apache.spark.mllib.evaluation._
    import org.apache.spark.mllib.tree._
    import org.apache.spark.mllib.regression.LabeledPoint
    import org.apache.spark.mllib.linalg.Vectors
    import org.apache.spark.mllib.tree.model._
    import org.apache.spark.rdd._
    import org.apache.spark.mllib.util.MLUtils
    import org.apache.spark.mllib.classification.LogisticRegressionModel


    object PredictOOS {
    def getMetrics(model: DecisionTreeModel, data: RDD[LabeledPoint]):
        MulticlassMetrics = {
      val predictionsAndLabels = data.map(example =>
        (model.predict(example.features), example.label)
      )
      new MulticlassMetrics(predictionsAndLabels)
    }

    def main(args: Array[String]) {
        val conf = new SparkConf().setAppName("Predict OOS")
        val spark = new SparkContext(conf)

        val data = spark.textFile("D:/data/g1-svm.csv")
        val parsedData = data.map { line =>
        val parts = line.split(',').map(_.toDouble)
        LabeledPoint(parts(0), Vectors.dense(parts.tail))
        }
        val splits = parsedData.randomSplit(Array(0.8, 0.2), seed = 11L)
        val training = splits(0).cache()
        val test = splits(1)

        val model = DecisionTree.trainClassifier(training, 2, Map[Int,Int]    (), "gini", 20, 300)

        val metrics = getMetrics(model, test)

        println(" confusionMatrix is generated")
        spark.stop()
  }
}

在

D:\ScalaApps\sparklr>cd ../oos

D:\ScalaApps\oos>sbt
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m; sup
port was removed in 8.0
[info] Set current project to Proj_oos (in build file:/D:/ScalaApps/oos/)
> compile
[info] Compiling 1 Scala source to D:\ScalaApps\oos\target\scala-2.11\classes...

[error] D:\ScalaApps\oos\src\main\scala\oos.scala:5: not found: type MulticlassM
etrics
[error]                     MulticlassMetrics = {
[error]                     ^
[error] D:\ScalaApps\oos\src\main\scala\oos.scala:4: not found: type DecisionTre
eModel
[error]                 def getMetrics(model: DecisionTreeModel, data: RDD[Label
edPoint]):
[error]                                       ^
[error] D:\ScalaApps\oos\src\main\scala\oos.scala:4: not found: type RDD
[error]                 def getMetrics(model: DecisionTreeModel, data: RDD[Label
edPoint]):
[error]                                                                ^
[error] D:\ScalaApps\oos\src\main\scala\oos.scala:9: not found: type MulticlassM
etrics
[error]                   new MulticlassMetrics(predictionsAndLabels)
[error]                       ^
[error] D:\ScalaApps\oos\src\main\scala\oos.scala:19: not found: value LabeledPo
int
[error]                         LabeledPoint(parts(0), Vectors.dense(parts.tail)
)
[error]                         ^
[error] D:\ScalaApps\oos\src\main\scala\oos.scala:19: not found: value Vectors
[error]                         LabeledPoint(parts(0), Vectors.dense(parts.tail)
)
[error]                                                ^
[error] D:\ScalaApps\oos\src\main\scala\oos.scala:25: not found: value DecisionT
ree
[error]                         val model = DecisionTree.trainClassifier(trainin
g, 2, Map[Int,Int](), "gini", 20, 300)
[error]                                     ^
[error] 7 errors found
[error] (compile:compileIncremental) Compilation failed
[error] Total time: 5 s, completed Dec 4, 2015 10:39:22 PM
>

请建议如果我错过任何东西。我被困在这部分编制，从很长的..Any帮助将不胜AP preciated

Please suggest if I am missing anything. I am stuck at this compiling part from very long ..Any help would be greatly appreciated

这是一个编辑到原来的职位。以上code编译成功，但是当我在写输出到文件失败了。

This is an edit to the original post. The above code compiled successfully but it failed when I was writing the output to a file.

    metrics.confusionMatrix.saveAsTextFile("D:/spark4/confMatrix2")

错误

D:\ScalaApps\oos>sbt
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m; sup
port was removed in 8.0
[info] Set current project to Proj_oos (in build file:/D:/ScalaApps/oos/)
> compile
[info] Compiling 1 Scala source to D:\ScalaApps\oos\target\scala-2.10\classes...

[error] D:\ScalaApps\oos\src\main\scala\oos.scala:44: value saveAsTextFile is no
t a member of org.apache.spark.mllib.linalg.Matrix
[error]                         metrics.confusionMatrix.saveAsTextFile("D:/spark
4/confMatrix2")
[error]                                                 ^
[error] one error found
[error] (compile:compileIncremental) Compilation failed
[error] Total time: 5 s, completed Dec 5, 2015 9:21:03 AM
>

有另一个包我需要进口的saveAsTextFile工作？

Is there another package I need to import for saveAsTextFile to work ?

推荐答案

您应该添加到您的build.sbt以下依赖性：

You should add to your build.sbt the following dependency:

libraryDependencies += "org.apache.spark" %% "spark-mllib" % "1.4.0"

而在你的斯卡拉文件添加以下导入：

And in your scala-file add the following import:

import org.apache.spark.{SparkConf, SparkContext}

希望这有助于

这篇关于斯卡拉code在SBT不编译的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！