如何使用 Avro 二进制编码器对 Kafka 消息进行编码/解码? [英] How to encode/decode Kafka messages using Avro binary encoder?

问题描述

我正在尝试使用 Avro 来读取/写入 Kafka 的消息.有没有人有使用 Avro 二进制编码器对将放入消息队列的数据进行编码/解码的示例?

I'm trying to use Avro for messages being read from/written to Kafka. Does anyone have an example of using the Avro binary encoder to encode/decode data that will be put on a message queue?

与 Kafka 部分相比，我更需要 Avro 部分.或者，也许我应该看看不同的解决方案?基本上，我试图找到一个更有效的 JSON 空间解决方案.刚才提到了 Avro，因为它可以比 JSON 更紧凑.

I need the Avro part more than the Kafka part. Or, perhaps I should look at a different solution? Basically, I'm trying to find a more efficient solution to JSON with regards to space. Avro was just mentioned since it can be more compact than JSON.

推荐答案

我终于想起去问 Kafka 邮件列表了，得到了以下答案，效果很好.

I finally remembered to ask the Kafka mailing list and got the following as an answer, which worked perfectly.

是的，您可以将消息作为字节数组发送.如果你看构造函数在 Message 类中，您将看到 -

Yes, you can send messages as byte arrays. If you look at the constructor of the Message class, you will see -

定义这个(字节:数组[字节])

def this(bytes: Array[Byte])

现在，看看 Producer send() API -

Now, looking at the Producer send() API -

def send(producerData: ProducerData[K,V]*)

def send(producerData: ProducerData[K,V]*)

您可以将 V 设置为 Message 类型，将 K 设置为您想要的密钥.如果您不关心使用密钥进行分区，则将其设置为 Message也输入.

You can set V to be of type Message and K to what you want your key to be. If you don't care about partitioning using a key, then set that to Message type as well.

谢谢，尼哈

这篇关于如何使用 Avro 二进制编码器对 Kafka 消息进行编码/解码?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！