在Apache NiFi中定义Apache Avro Schema全名 [英] Defining Apache Avro Schema fullname in Apache NiFi

问题描述

使用NiFi 1.7.1(使用Java Avro 1.8.1)并在AvroSchemaRegistry中，我试图定义一个架构，该架构在顶层具有字段name和app.name.根据Avro docs [1]，我假设我可以像普通"name": "app.name"一样定义全名，但遇到错误Illegal character in: app.name.确实，全名的名称部分不允许使用点，但根据文档显示:如果指定的名称包含点，则假定它是一个全名..."

然后我尝试使用名称空间字段.使用以下架构:

{
  "type": "record",
  "name": "nameRecord",
  "fields": [
    {
      "type": [
        "string",
        "null"
      ],
      "name": "name"
    },
    {
      "type": [
        "string",
        "null"
      ],
      "namespace": "app",
      "name": "name"
    }
  ]
}

我遇到此错误:Duplicate field name in record nameRecord: name type:UNION pos:1 and name type:UNION pos:0

最终，我希望能够为这样的记录(在JSON中)定义一个架构:

{
  "name": "Joe",
  "app.name": "NiFi"
}

[1] https://avro.apache.org/docs/1.8.1/spec.html#names

解决方案

根据

但是从NiFi 1.5.0开始(通过 NIFI-4612 ) ，您可以在app.name的限制.

Using NiFi 1.7.1 (which uses Java Avro 1.8.1) and in the AvroSchemaRegistry, I'm trying to define a schema which has the fields name and app.name at the top level. According to the Avro docs[1] I would assume that I could just define the fullname like normal "name": "app.name" but I hit the error Illegal character in: app.name. It's true that the name portion of the fullname does not allow dots but according to the docs: "If the name specified contains a dot, then it is assumed to be a fullname..."

I then tried using the namespace field. Using the following schema:

{
  "type": "record",
  "name": "nameRecord",
  "fields": [
    {
      "type": [
        "string",
        "null"
      ],
      "name": "name"
    },
    {
      "type": [
        "string",
        "null"
      ],
      "namespace": "app",
      "name": "name"
    }
  ]
}

I hit this error: Duplicate field name in record nameRecord: name type:UNION pos:1 and name type:UNION pos:0

Ultimately, I'd like to be able to define a schema for record like this (in JSON):

{
  "name": "Joe",
  "app.name": "NiFi"
}

[1] https://avro.apache.org/docs/1.8.1/spec.html#names

解决方案

According to the docs, namespaces are only supported for record, enum, and fixed types, and other fields must adhere to the "regular" naming conventions for which a period (.) is not a valid character.

However as of NiFi 1.5.0 (via NIFI-4612), you could specify the schema in an AvroSchemaRegistry, and set "Validate Field Names" to false. This should allow you to bypass the restriction of having a field's name be app.name.

这篇关于在Apache NiFi中定义Apache Avro Schema全名的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！