如何在Spark上将JSON字符串转换为数据帧 [英] how to convert json string to dataframe on spark
问题描述
我想在Spark上将下面的字符串变量转换为dataframe.
I want to convert string variable below to dataframe on spark.
val jsonStr = "{ "metadata": { "key": 84896, "value": 54 }}"
我知道如何从json文件创建数据框.
I know how to create dataframe from json file.
sqlContext.read.json("file.json")
但是我不知道如何从字符串变量创建数据框.
but I don't know how to create dataframe from string variable.
如何将json字符串变量转换为数据帧.
How can I convert json String variable to dataframe.
推荐答案
对于
对于 Spark 2.1.x :
提示:这正在使用 Hint: this is using 对于旧版本: 这篇关于如何在Spark上将JSON字符串转换为数据帧的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!val events = sc.parallelize("""{"action":"create","timestamp":"2016-01-07T00:01:17Z"}""" :: Nil)
val df = sqlContext.read.json(events)
sqlContext.read.json(jsonRDD: RDD[Stirng])
重载.
还有sqlContext.read.json(path: String)
,它可以直接读取Json文件.
sqlContext.read.json(jsonRDD: RDD[Stirng])
overload.
There is also sqlContext.read.json(path: String)
where it reads a Json file directly. val jsonStr = """{ "metadata": { "key": 84896, "value": 54 }}"""
val rdd = sc.parallelize(Seq(jsonStr))
val df = sqlContext.read.json(rdd)