Spring Kafka-什么时候在后台调用Consumer.poll()? [英] Spring Kafka- When is exactly Consumer.poll() called behind the hood?

问题描述

我有一个Spring Boot应用程序，其中有一个Kafka Consumer.

I have a spring boot application in which I have single Kafka Consumer.

我正在使用具有默认使用者配置的DefaultKafkaConsumerFactory.我有一个ConcurrentListenerContainerFactory，并发性设置为1，我有一个用@KafkaListener注释的方法.

I am using a DefaultKafkaConsumerFactory with default Consumer Configurations. I have a ConcurrentListenerContainerFactory with concurrency set to 1, and I have a method annotated with @KafkaListener.

我正在听一个带有3个分区的主题，我有3个这样的使用者分别部署在不同的应用程序中.因此，每个消费者都在监听一个分区.

I am listening to a topic with 3 partitions and I have 3 of such consumers deployed each in different applications. Hence, each consumer is listening to one partition.

可以说，对消费者的民意调查是在幕后进行的，并提取了40条记录.然后是每个记录，依次提供给以@KafkaListener注释的方法，即提供记录1，等到方法完成处理，提供记录2，等到方法完成处理等等.发生上述情况还是针对获得的每条记录创建一个单独的线程，并且方法调用发生在单独的线程上，因此主线程不会阻塞，并且可以更快地轮询记录.

Lets say poll on the consumer is called under the hood and 40 records are fetched. Then is each record, provided to the method annotated with @KafkaListener serially i.e. record 1 provided, wait till method finishes processing, record 2 provided , wait till method finishes processing and so on. Does the above happen, or for every record obtained , a separate thread is created and the method invocation happens on a separate thread, so the main thread does not block and it can poll for records more quickly.

我还想更清楚地了解什么是消息侦听器容器以及最终的消息侦听器.

I would also like more clarity on what a message listener container is and the eventual message listener.

先谢谢您.

推荐答案

在1.3及更高版本中，每个使用者都只有一个线程；侦听器处理完前一个轮询的最后一条消息后，将执行下一个 poll().

In 1.3 and above there is a single thread per consumer; the next poll() is performed after the last message from the previous poll has been processed by the listener.

在早期版本中，有两个线程，而侦听器线程正在处理第一批时，执行了第二次(可能是第三次)轮询.这样做是为了避免由于侦听器速度慢而导致的重新平衡.线程模型非常复杂，我们不得不在必要时暂停/恢复使用者.KIP-62解决了重新平衡问题，因此我们能够使用当今使用的简单得多的线程模型.

In earlier versions, there were two threads and a second (and possibly third) poll was performed while the listener thread is processing the first batch. This was required to avoid a rebalance due to a slow listener. The threading model was very complicated and we had to pause/resume the consumer when necessary. KIP-62 fixed the rebalance problem so we were able to use the much simpler threading model in use today.

这篇关于Spring Kafka-什么时候在后台调用Consumer.poll()?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！