如何在kafka中定义多个序列化器? [英] How to define multiple serializers in kafka?

问题描述

说，我发布并使用了不同类型的Java对象，对于每个对象我都必须定义自己的序列化程序实现. 我们如何在kafka消费者/生产者属性文件中的"serializer.class"属性下提供所有实现?

Say, I publish and consume different type of java objects.For each I have to define own serializer implementations. How can we provide all implementations in the kafka consumer/producer properties file under the "serializer.class" property?

推荐答案

我们具有类似的设置，在不同主题中具有不同的对象，但在一个主题中始终具有相同的对象类型.我们使用 ByteArrayDeserializer Java API 0.9.0.1附带，这意味着或消息使用者仅获得byte[]作为消息的值部分(我们始终将String用作键).特定于主题的消息使用者所做的第一件事是调用正确的反序列化器以转换byte[].您可以使用 apache公地帮手班.很简单.

We have a similar setup with different objects in different topics, but always the same object type in one topic. We use the ByteArrayDeserializer that comes with the Java API 0.9.0.1, which means or message consumers get only ever a byte[] as the value part of the message (we consistently use String for the keys). The first thing the topic-specific message consumer does is to call the right deserializer to convert the byte[]. You could use a apache commons helper class. Simple enough.

如果您希望让KafkaConsumer为您进行反序列化，那么您当然可以编写自己的

If you prefer to let the KafkaConsumer do the deserialization for you, you can of course write your own Deserializer. The deserialize method you need to implement has the topic as the first argument. Use it as a key into a map that provides the necessary deserializer and off you go. My hunch is that in most cases you will just do a normal Java deserialization anyway.

第二种方法的缺点是您需要所有消息对象都具有一个公共的超类，以便能够正确地参数化ConsumerRecord<K,V>.但是，对于第一种方法，它仍然是ConsumerRecord<String, byte[]>.但是然后，您将byte[]转换为恰好在正确的位置需要的对象，并且在那里只需要进行一次投射即可.

The downside of the 2nd approach is that you need a common super class for all your message objects to be able to parameterize the ConsumerRecord<K,V> properly. With the first approach, however, it is ConsumerRecord<String, byte[]> anyway. But then you convert the byte[] to the object you need just at the right place and need only one cast right there.

这篇关于如何在kafka中定义多个序列化器?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！