如果使用者持有消息的时间长于自动提交间隔时间，kafka是否会丢失消息? [英] Does kafka lose message if consumer holds message longer then auto commit interval time?

问题描述

如果自动提交间隔时间为30秒，则消费者由于某些原因无法处理该消息并将其保留超过30秒，然后崩溃.自动提交偏移量机制是否会在消费者崩溃之前立即提交此偏移量?

Say if auto-commit interval time is 30 seconds, consumer for some reasons could not process the message and hold it longer than 30 seconds then crash. does the auto-commit offset mechanism commits this offset anyway right before consumer crash?

如果我的假设是正确的，那么消息会因为提交的偏移量而丢失，但是消息本身尚未得到处理?

If my assumption is correct, the message is lost as its offset committed but the message itself has not been processed?

推荐答案

让我们考虑您的使用者组名称为Test，并且在使用者组中只有一个使用者.

Lets consider your Consumer group name is Test and you have a single consumer in the Consumer Group.

启用自动提交后，仅在poll()调用期间和使用者关闭期间才提交偏移量.

When Auto-Commit is enabled, offsets are committed only during poll() calls and during closing of a consumer.

例如-auto.commit.interval.ms为5秒，而对poll()的每次调用都需要7秒.每次调用poll()时，都会检查自动提交间隔是否已经过去，是否已经过去(如上例所示)，它将提交偏移量.

For example- auto.commit.interval.ms is 5 secs, and every call to poll() takes 7 secs. When making every call to poll(), it will check if the auto commit interval has elapsed, if it has, like in the above example, it will commit the offset.

抵销在关闭使用者期间也会发生.

Offsets are also committed during closing of a consumer.

从文档-

关闭使用者，等待任何需要清除的默认超时时间为30秒.如果启用了自动提交，则将在默认超时范围内尽可能地提交当前偏移量."

"Close the consumer, waiting for up to the default timeout of 30 seconds for any needed cleanup. If auto-commit is enabled, this will commit the current offsets if possible within the default timeout".

您可以在此处详细了解-

You can read more about it here -

https://kafka.apache.org/10/javadoc/index.html?org/apache/kafka/clients/consumer/KafkaConsumer.html

现在，在您的问题上，如果没有再次调用poll()或没有关闭使用者，它将不会提交偏移量.

Now, onto your question, if poll() is not called again or consumer is not closed, it won't commit the offset.

这篇关于如果使用者持有消息的时间长于自动提交间隔时间，kafka是否会丢失消息?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！