使用 apache nifi 使用预定义文件将列添加到 csv [英] Add column to csv using predefined file using apache nifi

问题描述

我得到一个看起来像这样的原始 csv 文件

id,name,star1,sachith,22,那拉卡,13,abc,3

我想用另一个文件映射星形列

1 1S2 3S3 5S

最后 csv 应该看起来像

id,name,star,level1,sachith,2,3S2,那拉卡,1,1S3,abc,3,5S

我使用了

解决方案

使用

生成流文件:

更新记录:

配置 CSVReader 以将第一行视为标题.保持其他属性不变.配置 CSVRecordSetWrite 以将第一行视为标题，从架构文本属性派生架构并将架构文本设置为:

<代码>{"类型":"记录","name":"foobar","namespace":"my.example",领域":[{"姓名":"姓名",类型":字符串"},{姓名年龄"，类型":整数"},{"name":"id",类型":字符串"},{"name":"尼克",类型":字符串"}]}

请注意，它包含新列.ReplaceTextWithMapping:

映射文件内容:

1 1S2 3S3 4S

值由制表符分隔.正则表达式必须匹配每行中没有后跟逗号的最后一个值:

[0-9](?!,)

I get a raw csv file which looks like this

id,name,star
1,sachith,2
2,nalaka,1
3,abc,3

I want to map star column with another file where it has

1  1S
2  3S
3  5S

and finally csv should look like

id,name,star,level
1,sachith,2,3S
2,nalaka,1,1S
3,abc,3,5S

I have used ReplaceTextWithMapping, but it replaces all the 1,2,3 values including in id column.

Here it defines replacing a value, but I want to map and add a new column to the record.

Edit:

After @Upvote's answer. My ReplaceTextWithMapping conf

解决方案

Use ReplaceTextWithMapping. Overall flow:

GenerateFlowFile:

UpdateRecord:

Configure CSVReader to treat first line as header. Leave other properties untouched. Configure CSVRecordSetWrite to treat first line as header, schema to be derived from schema text property and set schema text to:

{
   "type":"record",
   "name":"foobar",
   "namespace":"my.example",
   "fields":[
      {
         "name":"name",
         "type":"string"
      },
      {
         "name":"age",
         "type":"int"
      },
      {
         "name":"id",
         "type":"string"
      },
      {
         "name":"nick",
         "type":"string"
      }
   ]
}

Notice that it includes the new column. ReplaceTextWithMapping:

Mapping file content:

1   1S
2   3S
3   4S

Values are separated by tab. Regex must match the last value not followed by a comma in each line:

[0-9](?!,)

Debuggex Demo

Result:

这篇关于使用 apache nifi 使用预定义文件将列添加到 csv的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！