Apache Kafka 客户端什么时候会抛出“Batch Expired"?例外? [英] When does the Apache Kafka client throw a "Batch Expired" exception?

问题描述

使用 Apache Kafka Java 客户端 (0.9)，我尝试使用 Kafka Producer 类.

Using the Apache Kafka Java client (0.9), I'm trying to send a long series of records to the broker using the Kafka Producer class.

异步send 方法 立即返回一段时间，然后在短时间内开始阻塞每次调用.大约三十秒后，客户端开始抛出异常 (TimeoutException)，带有消息批量过期".

The asynchronous send method returns immediately for a while, then starts blocking on each call for a short time period. After around thirty seconds, the client starts throwing exceptions (TimeoutException), with the message "Batch expired".

什么情况会导致抛出这个异常?

What circumstances cause this exception to be thrown?

推荐答案

此异常表示您正在以比发送记录更快的速度排队记录.

This exception indicates you are queueing records at a faster rate than they can be sent.

当您调用 send 方法，ProducerRecord 将存储在内部缓冲区中以发送给代理.一旦 ProducerRecord 被缓冲，该方法立即返回，无论是否已发送.

When you call the send method, the ProducerRecord will be stored in an internal buffer for sending to the broker. The method returns immediately once the ProducerRecord has been buffered, regardless of whether it has been sent.

记录被分组为批次以发送给代理，以减少每条消息的传输监听并提高吞吐量.

Records are grouped into batches for sending to the broker, to reduce the transport overheard per message and increase throughput.

将记录添加为批次后，发送该批次有时间限制，以确保它已在指定的持续时间内发送.这由 Producer 配置参数控制，request.timeout.ms，默认为到三十秒.

Once a record is added a batch, there is a time limit for sending that batch to ensure it has been sent within a specified duration. This is controlled by the Producer configuration parameter, request.timeout.ms, which defaults to thirty seconds.

如果批处理的排队时间超过超时限制，则会抛出异常.该批次中的记录将从发送队列中删除.

If the batch has been queued longer than the timeout limit, the exception will be thrown. Records in that batch will be removed from the send queue.

使用配置参数增加超时限制将允许客户端在到期前将批次排队更长时间.

这篇关于Apache Kafka 客户端什么时候会抛出“Batch Expired"?例外?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！