Kafka:如何获取主题的上次修改时间，即上一条消息添加到该主题的任何分区 [英] Kafka: How to get last modified time for a topic i.e. last message added to any partition of the topic

问题描述

我们的用例是从kafka中删除陈旧/未使用的主题，即，如果某个主题(在所有分区上)在过去7天中没有任何新消息，则我们将其视为陈旧/未使用并删除.

Our use case is to delete stale/unused topics from kafka i.e. if a topic (on all partitions) doesn't have any new message in last 7 days then we would consider it as stale/unused and delete it.

许多google结果建议将时间戳添加到邮件中，然后对其进行解析.对于新主题表示灵魂化将起作用的消息，但我们现有的主题&邮件中没有任何时间戳.

Many google results suggested to add timestamp to messages and then parse it. For new topics & messages that soultion would work but our existing topics & messages doesn't have any timestamp in them.

如何使它正常工作?

推荐答案

kafka.api.OffsetRequest.LatestTime()将返回添加到队列中的最新消息.您可以使用Simple Consumer API确定要读取的偏移量.

kafka.api.OffsetRequest.LatestTime() will return the latest message added to a queue. You can use the Simple Consumer API to determine which offset to read from.

有关更多详细信息，请参见 wiki 页面

For more details take look at the wiki page

这篇关于Kafka:如何获取主题的上次修改时间，即上一条消息添加到该主题的任何分区的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！