kafka最早和最新偏移值之间的区别是什么 [英] What is the difference between kafka earliest and latest offset values

问题描述

producer发送消息1，2，3，4

producer sends messages 1, 2, 3, 4

consumer接收消息1，2，3，4

consumer receives messages 1, 2, 3, 4

consumer崩溃/断开连接

producer发送消息5、6、7

producer sends messages 5, 6, 7

consumer会重新启动，并且应该接收从5而不是7开始的消息

consumer comes back up and should receive messages starting from 5 instead of 7

对于这种结果，我必须使用哪个offset值以及需要做哪些其他更改/配置

For this kind of result, which offset value I have to use and what are the other changes/configurations need to do

推荐答案

当使用者加入使用者组时，它将获取最后一个提交的偏移量，因此如果在崩溃之前提交了最新的偏移量，它将重新从5、6、7读取.偏移量(so 4). auto.offset.reset属性的earliest和latest值在使用者启动时使用，但分配的分区没有提交的偏移量.在这种情况下，您可以选择是否要从头开始(最早)或在最后一条之后(最新)重新读取所有消息.

When a consumer joins a consumer group it will fetch the last committed offset so it will restart to read from 5, 6, 7 if before crashing it committed the latest offset (so 4). The earliest and latest values for the auto.offset.reset property is used when a consumer starts but there is no committed offset for the assigned partition. In this case you can chose if you want to re-read all the messages from the beginning (earliest) or just after the last one (latest).

这篇关于kafka最早和最新偏移值之间的区别是什么的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！