Kafka:如何获取主题的上次修改时间，即添加到主题任何分区的最后一条消息 [英] Kafka: How to get last modified time for a topic i.e. last message added to any partition of the topic

问题描述

我们的用例是从 kafka 中删除陈旧/未使用的主题，即如果一个主题(在所有分区上)在过去 7 天内没有任何新消息，那么我们会将其视为陈旧/未使用并删除它.

Our use case is to delete stale/unused topics from kafka i.e. if a topic (on all partitions) doesn't have any new message in last 7 days then we would consider it as stale/unused and delete it.

许多谷歌结果建议在消息中添加时间戳，然后对其进行解析.对于新主题 &消息，soultion 会起作用，但我们现有的主题 &消息中没有任何时间戳.

Many google results suggested to add timestamp to messages and then parse it. For new topics & messages that soultion would work but our existing topics & messages doesn't have any timestamp in them.

我怎样才能让它工作?.

How can I get this working ?.

推荐答案

kafka.api.OffsetRequest.LatestTime() 将返回添加到队列的最新消息.您可以使用 Simple Consumer API 来确定要读取的偏移量.

kafka.api.OffsetRequest.LatestTime() will return the latest message added to a queue. You can use the Simple Consumer API to determine which offset to read from.

有关详细信息，请查看 wiki页

For more details take look at the wiki page

这篇关于Kafka:如何获取主题的上次修改时间，即添加到主题任何分区的最后一条消息的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！