如何获取kafka主题分区的最后/结束偏移量? [英] How can I get the last/end offset of a kafka topic partition?
问题描述
我正在使用Java编写kafka
使用者.我想保持消息的实时性,因此,如果有太多消息在等待使用,例如1000条或更多,则应该放弃未使用的消息,并从最后一个偏移量开始使用.
I'm writing a kafka
consumer using Java. I want to keep the real time of the message, so if there are too many messages waiting for consuming, such as 1000 or more, I should abandon the unconsumed messages and start consuming from the last offset.
对于此问题,我尝试比较主题的最后提交的偏移量和主题的结束偏移量(仅1个分区),如果这两个偏移量之间的差大于某个值,则将设置将该主题作为下一个偏移量,以便我可以放弃这些多余的消息.
For this problem, I try to compare the last committed offset and the end offset of a topic(only 1 partition), if the difference between these two offsets is larger than a certain amount, I will set the last committed offset of the topic as next offset so that I can abandon those redundant messages.
现在我的问题是如何获得主题的结尾偏移,有人说我可以使用旧的使用者,但是它太复杂了,新使用者是否具有此功能?
Now my problem is how to get the end offset of a topic, some people say I can use old consumer, but it's too complicated, do new consumer has this function?
推荐答案
新消费者也很复杂.
//assign the topic
consumer.assign();
//assign the topic
consumer.assign();
//seek to end of the topic
consumer.seekToEnd();
//seek to end of the topic
consumer.seekToEnd();
//the position is the latest offset
consumer.position();
//the position is the latest offset
consumer.position();
这篇关于如何获取kafka主题分区的最后/结束偏移量?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!