Apache Kafka客户端什么时候抛出“Batch Expired”例外？ [英] When does the Apache Kafka client throw a "Batch Expired" exception?

问题描述

使用Apache Kafka Java客户端（0.9），我正在尝试使用 Kafka Producer类。

Using the Apache Kafka Java client (0.9), I'm trying to send a long series of records to the broker using the Kafka Producer class.

异步 send方法立即返回，然后在短时间内开始阻止每次通话。大约三十秒后，客户端开始抛出异常（ TimeoutException ），消息批处理已过期。

The asynchronous send method returns immediately for a while, then starts blocking on each call for a short time period. After around thirty seconds, the client starts throwing exceptions (TimeoutException), with the message "Batch expired".

什么情况会导致抛出此异常？

What circumstances cause this exception to be thrown?

推荐答案

此异常表示您以比发送速度更快的速度对记录进行排队。

This exception indicates you are queueing records at a faster rate than they can be sent.

当您致电 发送 方法， ProducerRecord 将存储在内部缓冲区中，以便发送给代理。无论何时缓冲 ProducerRecord ，该方法都会立即返回是否已发送。

When you call the send method, the ProducerRecord will be stored in an internal buffer for sending to the broker. The method returns immediately once the ProducerRecord has been buffered, regardless of whether it has been sent.

将记录分组到批次中，以便发送给代理，减少每封邮件的传输，并提高吞吐量。

Records are grouped into batches for sending to the broker, to reduce the transport overheard per message and increase throughput.

一旦添加了一个批次的记录，发送该批次有一个时间限制，以确保它在指定的持续时间内发送。这由Producer配置参数 request.timeout.ms 控制，默认值为

Once a record is added a batch, there is a time limit for sending that batch to ensure it has been sent within a specified duration. This is controlled by the Producer configuration parameter, request.timeout.ms, which defaults to thirty seconds.

如果批处理的排队时间超过了超时限制，则会抛出异常。该批次中的记录将从发送队列中删除。

If the batch has been queued longer than the timeout limit, the exception will be thrown. Records in that batch will be removed from the send queue.

使用配置参数增加超时限制将允许客户端在到期前将批次排队更长时间。

这篇关于Apache Kafka客户端什么时候抛出“Batch Expired”例外？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！