在 nifi 中将 JSON 转换为 CSV [英] Convert JSON to CSV in nifi

问题描述

我想在 nifi 中将 JSON 文件转换为 CSV.我们可以用 Python 和其他编程语言来实现这一点，并且有很多关于它的文章.我有多个 JSON 文件，每个文件都有不同的架构(一个特定文件只有一个架构).我可以看到有将 CSV 转换为 JSON 和其他转换的模板.但是我没有看到任何将 JSON 数据转换为 CSV 的模板.我已经阅读了文章 https://community.hortonworks.com/articles/64069/converting-a-large-json-file-into-csv.html ，但是这里我们对架构进行了硬编码.由于我有多个文件并且每个文件都有不同的架构，因此我无法对架构进行硬编码.请提出任何建议.

I want to convert JSON files to CSV in nifi. We can achieve this in Python and other programming languages and have multiple articles on it. I have multiple JSON files and each file has different schema(one specific file will have one schema only). I can see there are templates to convert CSV to JSON and other conversions. But I didn't see any template to convert JSON data to CSV. I have gone through the article https://community.hortonworks.com/articles/64069/converting-a-large-json-file-into-csv.html ,however here we are hard coding the schema. As I have multiple files and each file has different schema, I can't hardcode the schema. Any suggestions please.

推荐答案

格式之间的转换通常通过 ConvertRecord 通过插入适当的记录读取器和记录写入器(在本例中为 JSON 读取器和 CSV 写入器)来完成.

Conversion between formats is typically done through ConvertRecord by plugging in the appropriate record reader and record writer, in this case a JSON reader and CSV writer.

要使用记录处理器，您需要为数据定义 Avro 模式并将它们放入模式注册表中，NiFi 提供了本地模式.

To make use of the record processors you need to defined Avro schemas for your data and put them in a schema registry, NiFi provides a local one.

有很多关于记录内容的示例和帖子，此幻灯片显示了 CSV 到 JSON 的示例，但很容易在您的场景中扭转这种情况:

There are lots of examples and posts out there about the record stuff, this slide deck shows an example of CSV to JSON, but would be easy to reverse the situation for your scenario:

https://www.slideshare.net/BryanBende/apache-nifi-记录处理

这篇文章还有一些其他信息:https://bryanbende.com/development/2017/06/20/apache-nifi-records-and-schema-registries

This post has some other info: https://bryanbende.com/development/2017/06/20/apache-nifi-records-and-schema-registries

这篇关于在 nifi 中将 JSON 转换为 CSV的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！