Kafka 0.10.0.0和更高版本的session.timeout.ms和max.poll.interval.ms之间的差异 [英] Difference between session.timeout.ms and max.poll.interval.ms for Kafka 0.10.0.0 and later versions
问题描述
我不清楚为什么我们同时需要session.timeout.ms和max.poll.interval.ms以及何时使用一个或多个?似乎这两个设置都指示协调器在假定死机之前等待从消费者那里获取心跳的时间上限.
I am unclear why we need both session.timeout.ms and max.poll.interval.ms and when would we use one or the other or both? It seems like both settings indicate the upper bound on the time the coordinator will wait to get the heartbeat from a consumer before assuming it's dead.
对于基于推荐答案
在KIP-62之前，只有session.timeout.ms(即Kafka 0.10.0和更早版本).通过0.10.1的一部分).
Before KIP-62, there is only session.timeout.ms (ie, Kafka 0.10.0 and earlier). max.poll.interval.ms is introduced via KIP-62 (part of Kafka 0.10.1).
KIP-62通过后台心跳线程将心跳与呼叫与poll()的耦合解耦，从而允许比心跳间隔更长的处理时间(即，两个连续的poll()之间的时间).
KIP-62, decouples heartbeats from calls to poll() via a background heartbeat thread, allowing for a longer processing time (ie, time between two consecutive poll()) than heartbeat interval.
假设处理一条消息需要1分钟.如果心跳和轮询耦合(即，在KIP-62之前)，则您需要将session.timeout.ms设置为大于1分钟，以防止消费者超时.但是，如果消费者死亡，则检测失败的消费者还需要花费超过1分钟的时间.
Assume processing a message takes 1 minute. If heartbeat and poll are coupled (ie, before KIP-62), you will need to set session.timeout.ms larger than 1 minute to prevent consumer to time out. However, if consumer dies, it also takes longer than 1 minute to detect the failed consumer.
KIP-62将轮询和心跳解耦，从而允许在两个连续的轮询之间发送心跳.现在您有两个线程正在运行，即心跳线线程和处理线程，因此，KIP-62为每个线程引入了超时. session.timeout.ms用于检测信号线程，而max.poll.interval.ms用于处理线程.
KIP-62 decouples polling and heartbeat allowing to sent heartbeat between two consecutive polls. Now you have two threads running, the heartbeat thread and the processing thread and thus, KIP-62 introduced a timeout for each. session.timeout.ms is for the heartbeat thread while max.poll.interval.ms is for the processing thread.
假定您设置了session.timeout.ms=30000，因此，消费者心跳线线程必须在此时间到期之前将心跳线发送给代理.另一方面，如果处理一条消息需要1分钟，则可以将max.poll.interval.ms设置为大于1分钟，以使处理线程有更多时间来处理消息.
Assume, you set session.timeout.ms=30000, thus, the consumer heartbeat thread must sent a heartbeat to the broker before this time expires. On the other hand, if processing of a single message takes 1 minutes, you can set max.poll.interval.ms larger than one minute to give the processing thread more time to process a message.
如果处理线程死亡，则需要max.poll.interval.ms进行检测.但是，如果整个使用者都死了(并且垂死的处理线程很可能使整个使用者(包括心跳线)崩溃，则只需session.timeout.ms即可检测到它.
If the processing thread dies, it takes max.poll.interval.ms to detect this. However, if the whole consumer dies (and a dying processing thread most likely crashes the whole consumer including the heartbeat thread), it takes only session.timeout.ms to detect it.
这个想法是，即使处理本身花费很长时间，也可以快速检测出失败的消费者.
The idea is, to allow for a quick detection of a failing consumer even if processing itself takes quite long.
这篇关于Kafka 0.10.0.0和更高版本的session.timeout.ms和max.poll.interval.ms之间的差异的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
