如何让 kafka 消费者从上次消费的偏移量而不是从头开始读取 [英] How to make kafka consumer to read from last consumed offset but not from beginning

问题描述

我是 kafka 的新手，并试图了解是否有办法从上次使用的偏移量读取消息，而不是从头读取消息.

I am new to kafka and trying to understand if there is a way to read messages from last consumed offset, but not from beginning.

我正在写一个示例案例，这样我的意图就不会偏离.

I am writing an example case, so that my intention will not get deviate.

Eg:
1) I produced 5 messages at 7:00 PM and console consumer consumed those.
2) I stopped consumer at 7:10 PM
3) I produced 10 message at 7:20 PM. No consumer had read those messages.
4) Now, i have started console consumer at 7:30 PM, without from-beginning.
5) Now, it Will read the messages produced after it has started. Not the earlier ones, which were produced at 7.20 PM

有没有办法获取从上次消费的偏移量中产生的消息.?

Is there a way to get the messages produced from last consumed offset.?

推荐答案

我是 kafka 的新手，并试图了解 是否有办法从上次使用的偏移量读取消息，而不是从头读取消息.

是的，可以使用控制台消费者从上次消费的偏移量中读取.您必须在调用 kafka-console-consumer 时添加 consumer.config 标志.

Yes, it is possible to use console consumer to read from the last consumed offset. You have to add consumer.config flag while invoking kafka-console-consumer.

示例:-

[root@sandbox bin]# ./kafka-console-consumer.sh --topic test1 --zookeeper localhost:2181 --consumer.config /home/mrnakumar/consumer.properties

此处/home/mrnakumar/consumer.properties 是一个包含group.id 的文件.这是/home/mrnakumar/consumer.properties 的外观:-

Here /home/mrnakumar/consumer.properties is a file containing group.id. Here is how the /home/mrnakumar/consumer.properties looks:-

group.id=consoleGroup

group.id=consoleGroup

使用consumer.config 时，可以从[ 使用--from-beginning] 开始或仅从日志的末尾读取.日志结束表示消费者启动后发布的所有消息.

Withoug using consumer.config, it is possible to read either from beginning [ by using --from-beginning] or end of the Log only. End of the Log means all the messages published after consumer start.

这篇关于如何让 kafka 消费者从上次消费的偏移量而不是从头开始读取的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！