如何将 DataFrame 转换为 Json? [英] How to convert DataFrame to Json?

问题描述

我有一个巨大的 Json 文件，其中的一小部分如下:

I have a huge Json file, a small part from it as follows:

{
    "socialNews": [{
        "adminTagIds": "",
        "fileIds": "",
        "departmentTagIds": "",
        ........
        ........
        "comments": [{
            "commentId": "",
            "newsId": "",
            "entityId": "",
            ....
            ....
        }]
    }]
    .....
    }

我在socialNews上应用了横向视图exlode如下:

I have applied lateral view exlode on socialNews as follows:

val rdd = sqlContext.jsonFile("file:///home/ashish/test")
rdd.registerTempTable("social")
val result = sqlContext.sql("select * from social LATERAL VIEW explode(socialNews) social AS comment")

现在我想将此结果 (DataFrame) 转换回 json 并保存到文件中，但我找不到任何 scala api 来进行转换.有没有标准库可以做到这一点，或者有什么方法可以解决这个问题?

Now I want to convert back this result (DataFrame) to json and save into a file, but i am not able to find any scala api to do conversion. Is there any standard library to do this or some way to figure it out?

推荐答案

val result: DataFrame = sqlContext.read.json(path)
result.write.json("/yourPath")

方法 write 在 DataFrameWriter 并且应该可以在 DataFrame 对象上访问.只需确保您的 rdd 是 DataFrame 类型而不是已弃用的 SchemaRdd 类型.您可以显式提供类型定义 val data: DataFrame 或使用 toDF() 转换为 dataFrame.

The method write is in the class DataFrameWriter and should be accessible to you on DataFrame objects. Just make sure that your rdd is of type DataFrame and not of deprecated type SchemaRdd. You can explicitly provide type definition val data: DataFrame or cast to dataFrame with toDF().

这篇关于如何将 DataFrame 转换为 Json?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！