通过了解消息在Kafka中的分区和偏移量来检索消息 [英] Retrieve a message by knowing its partition and offset in Kafka

问题描述

我正在研究Kafka 0.9.我想知道是否有任何方法可以通过了解分区和偏移量来从其主题中检索已处理的消息.例如，使用者当前正在使用分区1和偏移量10处的消息.我想在同一分区和偏移量5处获取消息.

I'm working on Kafka 0.9. I'm wondering if there is any approach to retrieve a message, which has been processed, from its topic by knowing the partition and offset. For example, the consumer is currently consuming the message at partition 1 and offset 10. And I want to get the message at the same partition and offset 5.

我能想到的一种方法是将偏移量重置为5并消耗一条消息.但是poll()方法只能返回一批消息.因此，我必须接受第一个消息，而忽略其他消息.处理完消息后，将偏移量重置回去.

One way that I can think of is to reset the offset to 5 and consume one single message. But the poll() method can only return a batch of messages. So I have to take the first message and disregard the others. After processing the message, the offset is reset back.

我认为这会起作用.但是仍然想知道是否还有其他优雅的方法.

I think this will work. But still want to know if there is any other elegant way of doing it.

推荐答案

您应该能够使用搜索"方法从所需的偏移量中读取消息.

You should be able to use the "seek" method to read the message from the offset you require.

看看控制消费者的位置" https://kafka.apache.org/090 /javadoc/org/apache/kafka/clients/consumer/KafkaConsumer.html

Take a look at the "Controlling the Consumer's Position" https://kafka.apache.org/090/javadoc/org/apache/kafka/clients/consumer/KafkaConsumer.html

这篇关于通过了解消息在Kafka中的分区和偏移量来检索消息的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！