SPARK/SQL:spark无法将符号解析为DF [英] SPARK/SQL:spark can't resolve symbol toDF

问题描述

在我的项目中，我的外部库是spark-assembly-1.3.1-hadoop2.6.0，如果我按'.'，IDE会通知我toDF()，但它会告诉我在编码时无法解析符号toDF().抱歉，我在Apache Spark文档中找不到toDF().

In my project, my external library is spark-assembly-1.3.1-hadoop2.6.0, if I press '.', the IDE inform me toDF(), but it inform me that can't resolve symbol toDF() when I code it in. I'm sorry I can't find the toDF() in Apache Spark doc.

case class Feature(name:String, value:Double, time:String, period:String)
val RESRDD = RDD.map(tuple => {
    var bson=new BasicBSONObject();
    bson.put("name",name);
    bson.put("value",value);
    (null,bson);
})

RESRDD
 .map(_._2)
 .map(f => Feature(f.get("name").toString, f.get("value").toString.toDouble))
 .toDF()

推荐答案

要使用toDF，您必须先导入sqlContext.implicits:

To be able to to use toDF you have to import sqlContext.implicits first:

val sqlContext = new org.apache.spark.sql.SQLContext(sc)
import sqlContext.implicits._

case class Foobar(foo: String, bar: Integer)

val foobarRdd = sc.parallelize(("foo", 1) :: ("bar", 2) :: ("baz", -1) :: Nil).
    map { case (foo, bar) => Foobar(foo, bar) } 

val foobarDf = foobarRdd.toDF
foobarDf.limit(1).show

这篇关于SPARK/SQL:spark无法将符号解析为DF的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！