Apache Nifi-从Avro提取属性 [英] Apache Nifi - Extract Attributes From Avro

问题描述

我正在努力从Avro和JSON提取属性.我可以使用EvaluateJsonPath处理器从JSON中提取属性.我正在尝试在Avro上执行相同的操作，但不确定是否可以实现.

I'm trying to get my head around on extracting attributes from Avro and JSON. I'm able to extract attributes from JSON by using EvaluateJsonPath processor. I'm trying to do the same on Avro, but i'm not sure whether it is achievable.

这是我的流程，ExecuteSQL-> SplitAvro-> UpdateAttribute

Here is my flow, ExecuteSQL -> SplitAvro -> UpdateAttribute

UpdateAttribute是我要提取属性的处理器.请在下面找到UpdateAttribute处理器的快照，

UpdateAttribute is the processor where i want to extract the attributes. Please find below snapshot of UpdateAttribute processor,

所以，我的基本问题是，我们可以从Avro中提取属性吗?如果是，请为我提供正确的方法.还是必须在提取属性之前始终使用ConvertAvroToJSON?

So, my basic question is, could we extract attributes form Avro? If yes, please provide me the right approach. Or is it necessary to use ConvertAvroToJSON always before extracting the attributes?

推荐答案

当前， NiFi中无法直接从Avro中提取属性(有尚无AvroPath (例如XML的XPath或JSON的JsonPath)，因此您可以在提取属性之前使用ConvertAvroToJSON.

Currently, there is no way in NiFi to extract attributes directly from Avro (there is not yet an AvroPath like XPath for XML or JsonPath for JSON) so as you said you can use ConvertAvroToJSON before extracting the attributes.

或者，我编写了一个Groovy脚本供ExecuteScript处理器使用，它以"Avro path"值作为动态属性(每个都以avro.path开头，其值实际上是JsonPath)，并在其中将Avro转换为JSON.内存，需要您下载并指向Avro JAR.如果您有兴趣，可以在此处发布，但实际上唯一的好处是可以在Avro中维护流文件的内容，尽管这可能很烦人，但您可以使用ConvertAvroToJson-> EvaluateJsonPath-> ConvertJsonToAvro作为解决方法.

Alternatively, I wrote a Groovy script for use in an ExecuteScript processor, it takes "Avro path" values as dynamic properties (each starting with avro.path and whose value is really JsonPath), does the conversion of Avro to JSON in memory, and requires you download and point to the Avro JARs. I can post it here if you are interested, but really its only advantage is to maintain the flow file content in Avro, and although it might be annoying, you could use ConvertAvroToJson -> EvaluateJsonPath -> ConvertJsonToAvro as the workaround.

这篇关于Apache Nifi-从Avro提取属性的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！