Kafka10.1 heartbeat.interval.ms, session.timeout.ms 和 max.poll.interval.ms [英] Kafka10.1 heartbeat.interval.ms, session.timeout.ms and max.poll.interval.ms
问题描述
我正在使用 kafka 0.10.1.1 并与以下 3 个属性混淆.
I am using kafka 0.10.1.1 and confused with the following 3 properties.
heartbeat.interval.ms
session.timeout.ms
max.poll.interval.ms
heartbeat.interval.ms - 这是在 0.10.1 中添加的，它将在轮询之间发送心跳.session.timeout.ms - 如果没有对 kafka 的请求，则开始重新平衡，并且在每次轮询时都会重置.max.poll.interval.ms - 这是整个民意调查.
heartbeat.interval.ms - This was added in 0.10.1 and it will send heartbeat between polls. session.timeout.ms - This is to start rebalancing if no request to kafka and it gets reset on every poll. max.poll.interval.ms - This is across the poll.
但是，kafka 什么时候开始重新平衡?为什么我们需要这三个?所有这些的默认值是什么?
But, when does kafka starts rebalancing? Why do we need these 3? What are the default values for all of them?
谢谢
推荐答案
假设我们讨论的是 Kafka 0.10.1.0 或更高版本，其中每个消费者实例使用两个线程来运行.一个是调用 poll 的用户线程；另一个是心跳线程，专门处理心跳的事情.
Assuming we are talking about Kafka 0.10.1.0 or upwards where each consumer instance employs two threads to function. One is user thread from which poll is called; the other is heartbeat thread that specially takes care of heartbeat things.
session.timeout.ms 用于心跳线程.如果协调器在此时间间隔过去之前未能从消费者那里获得任何心跳，它会将消费者标记为失败并触发新一轮的重新平衡.
session.timeout.ms is for heartbeat thread. If coordinator fails to get any heartbeat from a consumer before this time interval elapsed, it marks consumer as failed and triggers a new round of rebalance.
max.poll.interval.ms 用于用户线程.如果消息处理逻辑过于繁重，成本超过这个时间间隔，协调器会明确让消费者离开该组，并触发新一轮的重新平衡.
max.poll.interval.ms is for user thread. If message processing logic is too heavy to cost larger than this time interval, coordinator explicitly have the consumer leave the group and also triggers a new round of rebalance.
heartbeat.interval.ms 用于让其他健康的消费者更快地意识到重新平衡.如果协调器触发重新平衡，其他消费者只能通过接收带有 REBALANCE_IN_PROGRESS 异常封装的心跳响应来知道这一点.心跳请求发送得越快，消费者就越快知道它需要重新加入组.
heartbeat.interval.ms is used to have other healthy consumers aware of the rebalance much faster. If coordinator triggers a rebalance, other consumers will only know of this by receiving the heartbeat response with REBALANCE_IN_PROGRESS exception encapsulated. Quicker the heartbeat request is sent, faster the consumer knows it needs to rejoin the group.
建议值:session.timeout.ms :相对较低的值，例如 10 秒.max.poll.interval.ms:基于您的处理要求heartbeat.interval.ms:相对较低的值，最好是session.timeout.ms
Suggested values:
session.timeout.ms : a relatively low value, 10 seconds for instance.
max.poll.interval.ms: based on your processing requirements
heartbeat.interval.ms: a relatively low value, better 1/3 of the session.timeout.ms
这篇关于Kafka10.1 heartbeat.interval.ms, session.timeout.ms 和 max.poll.interval.ms的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
