Apache Kafka:TimeoutException，然后没有任何效果 [英] Apache Kafka: TimeoutException and then nothing works‏

问题描述

我正在尝试将消息从生产者发送到另一个DC中的kafka节点.生产者和使用者都设置了默认的 0.10.0.0 配置，并且消息大小不是很小(大约500k).在大多数情况下，发送邮件时都会遇到以下例外情况:

I am trying to send a message from a producer to a kafka node in another DC. Both the producer and consumer are set with the default 0.10.0.0 configuration and the message sizes are not so small (around 500k). Most of the time when sending messages I am encountered with these exceptions:

org.apache.kafka.common.errors.TimeoutException:包含1条记录的批处理由于超时而过期，同时向代理人请求主题0的元数据org.apache.kafka.common.errors.TimeoutException:无法在配置的最大阻止时间60000毫秒内分配内存.

org.apache.kafka.common.errors.TimeoutException: Batch containing 1 record(s) expired due to timeout while requesting metadata from brokers for topic-0 org.apache.kafka.common.errors.TimeoutException: Failed to allocate memory within the configured max blocking time 60000 ms.

此后，不再传输任何消息(即使未调用其余消息的回调).

And after that no more messages get transferred (even the callback for remaining messages is not getting called).

推荐答案

根据Kafka文档:

小的批处理大小将使批处理变得不那么普遍，并且可能会降低吞吐量(批处理大小为零将完全禁用批处理).非常大的批处理大小可能会浪费一些内存，因为我们总是在预期其他记录的情况下分配指定批处理大小的缓冲区.

A small batch size will make batching less common and may reduce throughput (a batch size of zero will disable batching entirely). A very large batch size may use memory a bit more wastefully as we will always allocate a buffer of the specified batch size in anticipation of additional records.

设置 batch.size = 0 ，它将解决此问题.

Set batch.size = 0, it will resolve the issue.

这篇关于Apache Kafka:TimeoutException，然后没有任何效果的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！