如何使用Avro二进制编码器对Kafka消息进行编码/解码? [英] How to encode/decode Kafka messages using Avro binary encoder?

问题描述

我正在尝试使用Avro来读取和写入Kafka的邮件.有没有人举过一个使用Avro二进制编码器对将放入消息队列中的数据进行编码/解码的示例?

I'm trying to use Avro for messages being read from/written to Kafka. Does anyone have an example of using the Avro binary encoder to encode/decode data that will be put on a message queue?

与Kafka相比，我更需要Avro部分.或者，也许我应该考虑其他解决方案?基本上，我试图在空间方面找到一种更有效的JSON解决方案.刚刚提到了Avro，因为它可以比JSON紧凑.

I need the Avro part more than the Kafka part. Or, perhaps I should look at a different solution? Basically, I'm trying to find a more efficient solution to JSON with regards to space. Avro was just mentioned since it can be more compact than JSON.

推荐答案

我终于想起了向Kafka邮件列表提问，并得到了以下答复，效果很好.

I finally remembered to ask the Kafka mailing list and got the following as an answer, which worked perfectly.

是的，您可以将消息作为字节数组发送.如果您看一下构造函数 Message类的内容，您将看到-

Yes, you can send messages as byte arrays. If you look at the constructor of the Message class, you will see -

def this(bytes:Array [Byte])

def this(bytes: Array[Byte])

现在，查看生产者send()API-

Now, looking at the Producer send() API -

def send(producerData:ProducerData [K，V] *)

def send(producerData: ProducerData[K,V]*)

您可以将V设置为Message类型，将K设置为您想要的密钥. 如果您不关心使用键进行分区，则将其设置为Message 键入.

You can set V to be of type Message and K to what you want your key to be. If you don't care about partitioning using a key, then set that to Message type as well.

谢谢， 内哈

这篇关于如何使用Avro二进制编码器对Kafka消息进行编码/解码?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！