保留配置偏移量.retention.minutes和log.retention.minutes之间的差异 [英] Difference between retention configuration offsets.retention.minutes and log.retention.minutes

问题描述

以下两种保留配置之间有什么区别?

What is the difference between the following two retention configurations?

• offsets.retention.minutes
• log.retention.minutes

我不知道它有什么不同或相互联系.据我了解，一旦偏移量被删除，日志中的记录就无法访问，反之亦然.有什么我误会的东西吗?

I don't get how it differs or relate to each other. From my understanding, once the offset is removed, the record in the log is not accessible and vice versa. Is there something I have misunderstood?

推荐答案

偏移量是指向消费者已使用的最新消息的指针.因此，如果您阅读10条消息，则偏移量将移动10位. offsets.retention.minutes 允许您将偏移量移回开头，如果在设定的时间段内未更改的话.

The offset is a pointer to the most recent message that has been consumed by a consumer. So if you read 10 messages, the offset moves 10 places. offsets.retention.minutes allows you to move the offset back to the beginning if it isn't changed within a set period of time.

为使其形象化，我们假设在Kafka主题中将字母 a 放在 g (按顺序)中，都在不同的时间进行.在开始使用消息之前，偏移量指向最早的消息:

To visualise it, let's assume that we put the letters a to g (in that order) in a Kafka topic, all at different times. Before we start consuming the messages, the offset points to the oldest message:

OFFSET:    *
MESSAGES:  a b c d e f g

现在我们消耗3条消息( a ， b ， c )，因此偏移量会移动:

Now we consume 3 messages (a, b, c) so the offset moves:

OFFSET:          *
MESSAGES:  a b c d e f g

现在，让我们假设已经设置了 log.retention.minutes = 10 ，然后将 a 和 b 放入了主题11分钟以前，但是其他消息是在最近插入的.我们会看到:

Now let's pretend we've set log.retention.minutes=10, and we put a and b into the topic 11 minutes ago, but the other messages were inserted more recently. We'd see:

OFFSET:          *
MESSAGES:      c d e f g

现在，让我们设置 offsets.retention.minutes = 1 ，并假设距离上次消耗任何东西已有90秒.我们会看到:

Now let's set offsets.retention.minutes=1, and pretend it has been 90 seconds since we last consumed anything. We'd see:

OFFSET:        *  
MESSAGES:      c d e f g

因为 c 现在是该主题上最古老的消息(也是将要使用的第一条消息).

because c is now the oldest message on the topic (and the first that will be consumed).

这篇关于保留配置偏移量.retention.minutes和log.retention.minutes之间的差异的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！