怎么把DataFrame转换成Json? [英] How to convert DataFrame to Json?

问题描述

我有一个巨大的Json文件，其中一小部分如下:

I have a huge Json file, a small part from it as follows:

{
    "socialNews": [{
        "adminTagIds": "",
        "fileIds": "",
        "departmentTagIds": "",
        ........
        ........
        "comments": [{
            "commentId": "",
            "newsId": "",
            "entityId": "",
            ....
            ....
        }]
    }]
    .....
    }

我在socialNews上应用了侧面视图，如下所示:

I have applied lateral view exlode on socialNews as follows:

val rdd = sqlContext.jsonFile("file:///home/ashish/test")
rdd.registerTempTable("social")
val result = sqlContext.sql("select * from social LATERAL VIEW explode(socialNews) social AS comment")

现在我想将此结果(DataFrame)转换回json并保存到文件中，但是我找不到任何scala api进行转换. 是否有任何标准库可以执行此操作或以某种方式解决该问题?

Now I want to convert back this result (DataFrame) to json and save into a file, but i am not able to find any scala api to do conversion. Is there any standard library to do this or some way to figure it out?

推荐答案

val result: DataFrame = sqlContext.read.json(path)
result.write.json("/yourPath")

方法write在类

The method write is in the class DataFrameWriter and should be accessible to you on DataFrame objects. Just make sure that your rdd is of type DataFrame and not of deprecated type SchemaRdd. You can explicitly provide type definition val data: DataFrame or cast to dataFrame with toDF().

这篇关于怎么把DataFrame转换成Json?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！