从Kafka请求两个时间戳之间的消息 [英] Request messages between two timestamps from Kafka
问题描述
是否可以根据摄取消息的时间段来消耗来自Kafka的消息?
Is it possible to consume messages from Kafka based on a time period in which the messages were ingested?
示例:我希望所有消息吸收到今天的0900-1000(现在是1200)之间.
Example: I want all messages ingested to a topic between 0900-1000 today (and now it's 1200).
如果只有一种方法可以指定开始时间,那很好-我的使用者可以在到达结束时间后停止处理消息.
If there is only a way to specify a start time, that's fine - my consumer can stop processing messages once it reaches the end time.
我可以看到从给定偏移量请求消息,获取第一个可用偏移量和最早可用偏移量的方法,但是在给定时间之后不是所有消息.
I can see methods for requesting messages from a given offset, and for getting the first available offset, and for the earliest available offset, but not all messages after a given time.
推荐答案
您可以使用offsetsForTimes
方法,该方法返回时间戳大于或等于给定时间戳的偏移量.
有关官方文档的更多信息,请参见:
You could use the offsetsForTimes
method which returns you offset whose timestamp is greater or equal to the give timestamp.
More information on the official doc here :
获取偏移量后,您可以尝试使用它并从那里开始读取.
After getting the offset you can seek using it and starting to read from there.
这篇关于从Kafka请求两个时间戳之间的消息的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!