如何在kafka中定义多个序列化器? [英] How to define multiple serializers in kafka?

问题描述

比如说，我发布和使用不同类型的 java 对象.对于每个对象，我必须定义自己的序列化器实现.我们如何在serializer.class"属性下提供 kafka 消费者/生产者属性文件中的所有实现?

Say, I publish and consume different type of java objects.For each I have to define own serializer implementations. How can we provide all implementations in the kafka consumer/producer properties file under the "serializer.class" property?

推荐答案

我们对不同主题中的不同对象有类似的设置，但在一个主题中始终使用相同的对象类型.我们使用 ByteArrayDeserializer 随 Java API 0.9.0.1 一起提供，这意味着消息消费者只获得 byte[] 作为消息的值部分(我们一直使用 String代码>键).特定于主题的消息消费者所做的第一件事是调用正确的解串器来转换 byte[].您可以使用 apache公共帮助类.很简单.

We have a similar setup with different objects in different topics, but always the same object type in one topic. We use the ByteArrayDeserializer that comes with the Java API 0.9.0.1, which means or message consumers get only ever a byte[] as the value part of the message (we consistently use String for the keys). The first thing the topic-specific message consumer does is to call the right deserializer to convert the byte[]. You could use a apache commons helper class. Simple enough.

如果您更愿意让 KafkaConsumer 为您进行反序列化，您当然可以编写自己的 Deserializer.您需要实现的 deserialize 方法将主题作为第一个参数.将它用作地图的关键，提供必要的解串器，然后就可以了.我的预感是，在大多数情况下，您无论如何都会进行普通的 Java 反序列化.

If you prefer to let the KafkaConsumer do the deserialization for you, you can of course write your own Deserializer. The deserialize method you need to implement has the topic as the first argument. Use it as a key into a map that provides the necessary deserializer and off you go. My hunch is that in most cases you will just do a normal Java deserialization anyway.

第二种方法的缺点是，您需要为所有消息对象使用一个通用的超类，以便能够正确地参数化 ConsumerRecord.然而，对于第一种方法，无论如何它都是 ConsumerRecord.但是随后您将 byte[] 转换为您需要的对象，就在正确的位置，并且只需要在那里进行一次强制转换.

The downside of the 2nd approach is that you need a common super class for all your message objects to be able to parameterize the ConsumerRecord<K,V> properly. With the first approach, however, it is ConsumerRecord<String, byte[]> anyway. But then you convert the byte[] to the object you need just at the right place and need only one cast right there.

这篇关于如何在kafka中定义多个序列化器?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！