AVRO中的数据验证 [英] Data validation in AVRO

问题描述

我是AVRO的新手，请问这是一个简单的问题. 我有一个使用AVRO模式进行记录调用的用例.

I am new to AVRO and please excuse me if it is a simple question. I have a use case where I am using AVRO schema for record calls.

假设我有avro模式

{
    "name": "abc",
    "namepsace": "xyz",
    "type": "record",
    "fields": [
        {"name": "CustId", "type":"string"},
        {"name": "SessionId", "type":"string"},
     ]
}

现在输入是否喜欢

{
    "CustId" : "abc1234"
    "sessionID" : "000-0000-00000"
}

我想对这些字段使用一些正则表达式验证，并且仅当输入格式如上所示时才使用此输入.有没有办法在avro模式中指定包含正则表达式的表达式?

I want to use some regex validations for these fields and I want take this input only if it comes in particular format shown as above. Is there any way to specify in avro schema to include regex expression?

还有其他支持这种格式的数据序列化格式吗?

Any other data serialization formats which supports something like this?

推荐答案

您应该可以使用自定义逻辑类型.然后，您可以将正则表达式直接包含在架构中.

You should be able to use a custom logical type for this. You would then include the regular expressions directly in the schema.

例如，以下是在JavaScript中实现代码的方法:

For example, here's how you would implement one in JavaScript:

var avro = require('avsc'),
    util = require('util');

/**
 * Sample logical type that validates strings using a regular expression.
 *
 */
function ValidatedString(attrs, opts) {
  avro.types.LogicalType.call(this, attrs, opts);
  this._pattern = new RegExp(attrs.pattern);
}
util.inherits(ValidatedString, avro.types.LogicalType);

ValidatedString.prototype._fromValue = function (val) {
  if (!this._pattern.test(val)) {
    throw new Error('invalid string: ' + val);
  }
  return val;
};

ValidatedString.prototype._toValue = ValidatedString.prototype._fromValue;

以及如何使用它:

var type = avro.parse({
  name: 'Example',
  type: 'record',
  fields: [
    {
      name: 'custId',
      type: 'string' // Normal (free-form) string.
    },
    {
      name: 'sessionId',
      type: {
        type: 'string',
        logicalType: 'validated-string',
        pattern: '^\\d{3}-\\d{4}-\\d{5}$' // Validation pattern.
      }
    },
  ]
}, {logicalTypes: {'validated-string': ValidatedString}});

type.isValid({custId: 'abc', sessionId: '123-1234-12345'}); // true
type.isValid({custId: 'abc', sessionId: 'foobar'}); // false

您可以在此处.

对于Java实现，我相信您将需要查看以下类:

For the Java implementation, I believe you will want to look at the following classes:

• LogicalType ，您需要扩展的基础.
• Conversion ，进行数据的转换(或您的情况下的验证).
• LogicalTypes 和 Conversions ，一些示例现有的实施方式.
• TestGenericLogicalTypes ，可以为您提供有用的起点的相关测试.

这篇关于AVRO中的数据验证的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！