Kafka AVRO - 从长到日期时间的转换 [英] Kafka AVRO - conversion from long to datetime
问题描述
当我想发送包含 long 类型字段的 AVRO 消息时,出现以下错误:
I get the following error when I want to send an AVRO message which contains a field that has the type long:
Caused by: org.apache.kafka.common.errors.SerializationException: Error deserializing Avro message for id 61
Caused by: java.lang.ClassCastException: java.lang.Long cannot be cast to org.joda.time.DateTime
我使用 Confluent 3.2.0 和 Apache Spark 2.2.0.这个错误是在处理 AVRO 消息并在控制台中打印它们的 Spark 作业中抛出的.在 AVRO schema 中,对应的字段是这样定义的:
I use Confluent 3.2.0 and Apache Spark 2.2.0. This error is thrown in a Spark Job which processes AVRO messages and prints them in a console. In the AVRO schema, the corresponding field is defined like this:
{\"name\": \"event_time\", \"type\": { \"type\" : \"long\", \"logicalType\": \"timestamp-millis\"}}
在.avsc
文件生成的Java类中,字段定义如下:
In the Java class generated from the .avsc
file, the field is defined as below:
private DateTime event_time;
推荐答案
我在使用 Confluent 4.0.0 和 Avro 1.8.2 时遇到了类似的问题.我有一个试图将 long 转换为 DateTime 的流处理器.我通过添加正确的转换克服了这个问题.在开始任何处理逻辑之前,我使用了 Specific Data 静态实用程序类并添加正确的逻辑类型转换.
I encountered a similar issue using Confluent 4.0.0 and Avro 1.8.2. I had a stream processor that was attempting to convert a long to a DateTime. I overcame the issue by adding the correct conversion. Before I begin any processing logic, I used the Specific Data static utilities class and adding the correct Logical type conversion.
SpecificData.get().addLogicalTypeConversion(new TimeConversions.TimestampConversion());
这篇关于Kafka AVRO - 从长到日期时间的转换的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!