在 Apache nifi 中将两个模式合并为一个 [英] Merge two schemas into one in Apache nifi
问题描述
我正在尝试使用 Apache nifi 将两个 csv 文件合并为一个 json.两个 csv 是persons.csv,其中包含有关人员的信息:
I'm trying to merge two csv files into a json using Apache nifi. Two csv's are persons.csv containing information about people:
Id|Name|Surname
ABC-123|John|Smith
ABC-111|Allan|Wood
ABC-001|Grace|Kelly
第二个 csv 包含这些人参加过的活动列表:
And the second csv contains list of events these people have attended:
EId|PId|Date|Desc
1|ABC-123|2017-05-01|"Groove party"
2|ABC-111|2017-06-01|"Snack No. One"
3|ABC-123|2017-06-01|"The night out"
我正在使用一个流 (git hub 上的 Nifi 流):
I'm using a flow of (Nifi flow on git hub):
- 获取文件
- UpdateAttribute (schema.name)
- 拆分记录
- 提取文本
- UpdateAttribute(correlation.id、newschema)
- 漏斗
- 合并记录/合并内容
- 放置文件
尝试实现最终的json:
Trying to achieve final json:
{
"Person": {
"Id": "ABC-123",
"Name": "John",
"Surname": "Smith",
"Events": [{
"Date": "2017-05-01",
"Name": "Groove party"
}, {
"Date": "2017-06-01",
"Name": "The night out"
}]
}
}
但我不确定如何设置合并记录,或者如何在将内容合并为单个 json 后加入多个 csv 行.有没有办法做到这一点?
But I'm not sure how to set up Merge Record, or how to join multiple csv lines after Merge Content into a single json. Is there a way how to do it?
推荐答案
你实际上可以使用
1- ConvertRecord(CSV 到 JSON)- 在您的情况下使用 Avro Schema
1- ConvertRecord(CSV to JSON) - using Avro Schema in your case
CSV阅读器
JsonRecordSetWriter
JsonRecordSetWriter
AvroSchemaRegistry
AvroSchemaRegistry
{
"name": "person",
"namespace": "nifi",
"type": "record",
"fields": [
{"name": "Id" , "type" : "string"},
{"name": "Name" , "type" : "string"},
{"name": "Surname" , "type" : "string"}
]
}
- 2 - SplitJson
- 3 - EvaluateJson
- 4 - LookupAttribute(获取 Desc SimpleCsvFileLookupService)
- 5 - LookupAttribute(获取日期 SimpleCsvFileLookupService)
- 6 - AttriutetoJson
这篇关于在 Apache nifi 中将两个模式合并为一个的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!