如何将dataframe作为json数组写入文件? (斯卡拉) [英] How to write dataframe as json array to a file? (scala)

问题描述

我有一个数据框，我想将其作为json数组写入scala中的单个文件中.

I have a dataframe and I want to write it as json array into a single file in scala.

dataframe.coalesce(1).write.format("json").save(destDir)

输出1: 每行一行，其中每一行都是一个json

output 1: One row per line, where each row is a json

dataframe.toJSON.coalesce(1).write.format("json").save(destDir)

输出2: 与输出1相同，但每行上都有一个看起来很奇怪的json {value:{key1:value1，key2:value2，...}

output 2: same as output 1, but a weird looking json on each row {value: {key1:value1, key2:value2, ... }

printWriter.write(dataframe.toJSON.collect.mkString("[",",","]"))

输出3:

它将json数组写入本地路径. 如果该路径用于hdfs，则即使该路径+文件存在，它也会显示FileNotFound.

It writes an array of json to a local path. If the path is for hdfs it says FileNotFound, even if the path + file exist.

推荐答案

请尝试以下操作以获取Spark 2+版本

Try the following for version of Spark 2+

dataframe.coalesce(1).write.json(destDir)

这篇关于如何将dataframe作为json数组写入文件? (斯卡拉)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！