保留期后的卡夫卡偏移 [英] Kafka Offset after retention period

问题描述

我有一个带有1个分区的kafka主题.如果其中包含100条消息，则偏移量将为0.99.

I have a kafka topic with 1 partition. if it had 100 messages in it the offset would be from 0.99.

根据kafka保留政策，所有邮件将在指定期限后被清除.

According the kafka retention policy all of the messages will be wiped out after the specified period.

，一旦全部清除(保留期之后)，我将向该主题发送100条新消息.现在，消息的新偏移量将从何处开始?是从100还是从0开始?

and i am sending 100 new messages to the topic once all have been wiped out(after retention period). Now, where would the new offset of the message start from? is it From 100 or from 0??

我试图了解新的偏移量是100-199还是0-99?

I am trying to understand whether the new offsets will be 100-199 or 0-99?

推荐答案

Kafka通过删除满足谓词的日志段来尊重日志保留策略，并且它永远不会删除活动日志段，这意味着活动日志的基本偏移量段将始终保留.

Kafka honors the log retention policy by deleting log segments which satisfy the predicate and it will not never remove the active log segment which means the base offset of the active log segment will always be kept.

以您的示例为例，如果第一批100条消息都在同一个日志段中，则Kafka将创建一个基本偏移量为100的新日志段，并在日志保留期为200之后删除旧的段.时间飞逝.因此，一条新消息的起始偏移量为100.

As for your example, if the first batch of 100 messages are all in a same log segment, Kafka will create a new log segment with the base offset of 100 and delete the old segment after a log-retention-period amount of time passed. So a new message starts offset at 100.

这篇关于保留期后的卡夫卡偏移的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！