kafka流多少次调用poll()来从kafka主题中获取记录 [英] How many times kafka stream invokes poll() for fetching records from kafka topic

问题描述

我想进一步了解kafka stream processor.我想知道用于从kafka提取数据的kafka流处理器的frequency of polling是什么.

I am trying to understand kafka stream processor a bit more. I want to know what is the frequency of polling by a kafka stream processor for fetching the data from kafka.

据我所知，kafka流处理器在内部创建了一个kafka Consumer客户端，该客户端从kafka获取数据(并调用poll()).

As I understand kafka stream processor internally creates a kafka Consumer client which fetches the data from kafka (and it invokes poll()).

因此，第一次调用poll()时，下次再次调用它是要从kafka中获取数据吗?每秒是否发生多次?我怎么知道poll()被调用了多少次(用于从kafka中获取数据).

So when first time poll() is called, what is the next time it would be called again to fetch data from kafka? Does it happen many times per second ? How can I know how many times poll() has been called (for fetching data from kafka).

任何有助于理解这一点的方法都会有很大帮助.

Any help in understanding this would be of great help.

推荐答案

poll()的调用频率没有绝对数字.有多个因素影响poll()的调用频率.作为简化模型，您可以假设调用poll()，获取N条记录，然后在处理完所有N条记录后再次调用.

There is no absolute number on how often poll() is called. There are multiple factors that impact how often poll() is called. As a simplified model, you can assume that poll() is called, fetches N records, and is called again after all N records are processed.

Kafka Streams收集指标，这些指标可告诉您poll()的调用频率，因此您可以轻松地对其进行监视: https://docs.confluent.io/current/streams/monitoring.html

Kafka Streams collects metrics that tell you how often poll() is called though, so you can easily monitor it: https://docs.confluent.io/current/streams/monitoring.html

这篇关于kafka流多少次调用poll()来从kafka主题中获取记录的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！