kafka 流调用 poll() 以从 kafka 主题中获取记录的次数 [英] How many times kafka stream invokes poll() for fetching records from kafka topic

问题描述

我试图更多地了解 kafka 流处理器.我想知道 kafka 流处理器从 kafka 获取数据的轮询频率是多少.

I am trying to understand kafka stream processor a bit more. I want to know what is the frequency of polling by a kafka stream processor for fetching the data from kafka.

据我所知，kafka 流处理器在内部创建了一个 kafka 消费者客户端，它从 kafka 获取数据(并调用 poll()).

As I understand kafka stream processor internally creates a kafka Consumer client which fetches the data from kafka (and it invokes poll()).

那么当第一次 poll() 被调用时，下一次它会被再次调用以从 kafka 获取数据是什么?它每秒发生很多次吗?我怎么知道 poll() 被调用了多少次(用于从 kafka 获取数据).

So when first time poll() is called, what is the next time it would be called again to fetch data from kafka? Does it happen many times per second ? How can I know how many times poll() has been called (for fetching data from kafka).

对理解这一点的任何帮助都会有很大帮助.

Any help in understanding this would be of great help.

推荐答案

poll() 被调用的频率没有绝对数字.有多种因素会影响 poll() 的调用频率.作为一个简化模型，您可以假设 poll() 被调用，获取 N 条记录，并在处理完所有 N 条记录后再次调用.

There is no absolute number on how often poll() is called. There are multiple factors that impact how often poll() is called. As a simplified model, you can assume that poll() is called, fetches N records, and is called again after all N records are processed.

Kafka Streams 收集的指标会告诉您 poll() 的调用频率，因此您可以轻松监控它:https://docs.confluent.io/current/streams/monitoring.html

Kafka Streams collects metrics that tell you how often poll() is called though, so you can easily monitor it: https://docs.confluent.io/current/streams/monitoring.html

这篇关于kafka 流调用 poll() 以从 kafka 主题中获取记录的次数的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！