通过知道消息在 Kafka 中的分区和偏移量来检索消息 [英] Retrieve a message by knowing its partition and offset in Kafka

问题描述

我正在研究 Kafka 0.9.我想知道是否有任何方法可以通过知道分区和偏移量来从其主题中检索已处理的消息.例如，消费者当前正在使用分区 1 和偏移量 10 处的消息.而我想在同一分区和偏移量 5 处获取消息.

I'm working on Kafka 0.9. I'm wondering if there is any approach to retrieve a message, which has been processed, from its topic by knowing the partition and offset. For example, the consumer is currently consuming the message at partition 1 and offset 10. And I want to get the message at the same partition and offset 5.

我能想到的一种方法是将偏移量重置为 5 并消耗一条消息.但是 poll() 方法只能返回一批消息.所以我必须接受第一条消息而忽略其他消息.处理完消息后，偏移量被重置回来.

One way that I can think of is to reset the offset to 5 and consume one single message. But the poll() method can only return a batch of messages. So I have to take the first message and disregard the others. After processing the message, the offset is reset back.

我认为这会奏效.但是还是想知道有没有其他优雅的做法.

I think this will work. But still want to know if there is any other elegant way of doing it.

推荐答案

您应该能够使用seek"方法从您需要的偏移量读取消息.

You should be able to use the "seek" method to read the message from the offset you require.

看看控制消费者的立场"https://kafka.apache.org/090/javadoc/org/apache/kafka/clients/consumer/KafkaConsumer.html

Take a look at the "Controlling the Consumer's Position" https://kafka.apache.org/090/javadoc/org/apache/kafka/clients/consumer/KafkaConsumer.html

这篇关于通过知道消息在 Kafka 中的分区和偏移量来检索消息的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！