Kafka Producer 批量大小 [英] Kafka Producer batch size

问题描述

我有一个关于batch.size生产者配置的问题.

I have a question related to batch.size producer configuration.

当 batch.size 达到并且生产者应用线程发送更多数据时会发生什么?

What happens when batch.size has reached and the producer app thread sends more data?

线程是否会阻塞直到包含批处理的缓冲区中有可用空间?

Does the thread block till space becomes available in the buffer containing the batch?

推荐答案

batch.size 以总字节数而不是消息数来衡量批大小.它控制在向 Kafka 代理发送消息之前要收集多少字节的数据.在不超过可用内存的情况下，将其设置得尽可能高.默认值为 16384.

batch.size measures batch size in total bytes instead of the number of messages. It controls how many bytes of data to collect before sending messages to the Kafka broker. Set this as high as possible, without exceeding available memory. The default value is 16384.

当您使用 Producer.send() 时，您会填满生产者的缓冲区.当缓冲区已满时，生产者将缓冲区发送给 Kafka 代理并开始重新填充缓冲区.

When you use Producer.send(), you fill up buffers on the producer. When a buffer is full, the producer sends the buffer to the Kafka broker and begins to refill the buffer.

这篇关于Kafka Producer 批量大小的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！