如何在消费者组kafka中动态添加消费者 [英] How to dynamically add consumers in consumer group kafka

问题描述

我该如何知道何时必须扩大消费群体中的消费群体.如果有快速的生产者，消费者扩大规模的诱因是什么?

How should I know when i have to scale the consumer in consumer group . What are the triggers for the consumers to scale when there is a fast producer ?

推荐答案

一种简单的方法是获取使用者滞后(可以将其计算为提交的偏移量与Beginning_offset之差)，并且如果滞后是在最后一次计算的n倍增加，您可以扩大规模，反之亦然.您可能需要考虑一些极端情况，例如，如果消费者减少并且滞后会增加，并且自动缩放功能可能会产生更多线程/机器，那么这种情况会发生.

One straight forward approach would be to get the consumer lag(this can be computed as the difference between committed offset and beginning_offset) and if the lag computed in the last n times is increasing you can scale up and vice versa. You might've to consider some edge cases for example in case consumers have gone down and lag would be increasing and the auto-scaling function might spawn more threads/machines).

这篇关于如何在消费者组kafka中动态添加消费者的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！