Kafka10.1 heartbeat.interval.ms，session.timeout.ms和max.poll.interval.ms [英] Kafka10.1 heartbeat.interval.ms, session.timeout.ms and max.poll.interval.ms

问题描述

我正在使用kafka 0.10.1.1，并与以下3个属性混淆.

I am using kafka 0.10.1.1 and confused with the following 3 properties.

heartbeat.interval.ms
session.timeout.ms
max.poll.interval.ms

heartbeat.interval.ms -在0.10.1中添加的，它将在两次轮询之间发送心跳. session.timeout.ms -如果没有对kafka的请求，它将开始重新平衡，并且每次轮询都将其重置. max.poll.interval.ms -这是整个民意调查.

heartbeat.interval.ms - This was added in 0.10.1 and it will send heartbeat between polls. session.timeout.ms - This is to start rebalancing if no request to kafka and it gets reset on every poll. max.poll.interval.ms - This is across the poll.

但是，卡夫卡什么时候开始重新平衡?为什么我们需要这3个?所有这些的默认值是什么?

But, when does kafka starts rebalancing? Why do we need these 3? What are the default values for all of them?

谢谢

推荐答案

假设我们正在谈论的是Kafka 0.10.1.0或更高版本，其中每个使用者实例都使用两个线程来运行.一个是从中调用poll的用户线程；另一个是从中调用poll的用户线程.另一个是心跳线，专门处理心跳的事情.

Assuming we are talking about Kafka 0.10.1.0 or upwards where each consumer instance employs two threads to function. One is user thread from which poll is called; the other is heartbeat thread that specially takes care of heartbeat things.

session.timeout.ms用于心跳线.如果协调员在此时间间隔之前未能从消费者那里得到任何心跳，则会将消费者标记为失败并触发新一轮的重新平衡.

session.timeout.ms is for heartbeat thread. If coordinator fails to get any heartbeat from a consumer before this time interval elapsed, it marks consumer as failed and triggers a new round of rebalance.

max.poll.interval.ms用于用户线程.如果消息处理逻辑太重而无法花费大于此时间间隔的费用，则协调器将明确要求使用者离开组，并触发新一轮的重新平衡.

max.poll.interval.ms is for user thread. If message processing logic is too heavy to cost larger than this time interval, coordinator explicitly have the consumer leave the group and also triggers a new round of rebalance.

heartbeat.interval.ms用于使其他健康的消费者更快地意识到重新平衡.如果协调器触发重新平衡，则其他使用者将仅通过接收带有封装的REBALANCE_IN_PROGRESS异常的心跳响应来了解此信息.发送心跳请求的速度越快，消费者知道需要重新加入该组的速度就越快.

heartbeat.interval.ms is used to have other healthy consumers aware of the rebalance much faster. If coordinator triggers a rebalance, other consumers will only know of this by receiving the heartbeat response with REBALANCE_IN_PROGRESS exception encapsulated. Quicker the heartbeat request is sent, faster the consumer knows it needs to rejoin the group.

建议值:
session.timeout.ms:一个相对较低的值，例如10秒.
max.poll.interval.ms:根据您的处理要求
heartbeat.interval.ms:相​​对较低的值，最好是session.timeout.ms

Suggested values:
session.timeout.ms : a relatively low value, 10 seconds for instance.
max.poll.interval.ms: based on your processing requirements
heartbeat.interval.ms: a relatively low value, better 1/3 of the session.timeout.ms

这篇关于Kafka10.1 heartbeat.interval.ms，session.timeout.ms和max.poll.interval.ms的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！