如何在scala中将元组列表转换为数据框 [英] How to convert list of tuple to dataframe in scala

问题描述

我有一个字符串元组列表:List[(String, String, String)].

如何使用Scala将其转换为数据框?

解决方案

您创建SparkSession(从Spark 2.0.0开始)或SQLContext，然后可以使用隐式toDF():

Spark 1.6或更早版本:

val sc = new SparkContext("local", "test")
val sqlContext = new SQLContext(sc)
import sqlContext.implicits._

val df: DataFrame = list.toDF() // with default column names: _1, _2, _3
val dfWithColNames: DataFrame = list.toDF("col1", "col2", "col3")

Spark 2.0.0或更高版本:

val sparkSession: SparkSession = SparkSession.builder().appName("test").master("local").getOrCreate()
import sparkSession.implicits._

val df: DataFrame = list.toDF() // with default column names: _1, _2, _3
val dfWithColNames: DataFrame = list.toDF("col1", "col2", "col3")

I have a list of tuples of strings: List[(String, String, String)].

How can I convert it into dataframe with Scala?

解决方案

You create a SparkSession (as of Spark 2.0.0) or a SQLContext, and then you can use the implicit toDF():

Spark 1.6 or earlier:

val sc = new SparkContext("local", "test")
val sqlContext = new SQLContext(sc)
import sqlContext.implicits._

val df: DataFrame = list.toDF() // with default column names: _1, _2, _3
val dfWithColNames: DataFrame = list.toDF("col1", "col2", "col3")

Spark 2.0.0 or newer:

val sparkSession: SparkSession = SparkSession.builder().appName("test").master("local").getOrCreate()
import sparkSession.implicits._

val df: DataFrame = list.toDF() // with default column names: _1, _2, _3
val dfWithColNames: DataFrame = list.toDF("col1", "col2", "col3")

这篇关于如何在scala中将元组列表转换为数据框的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！