卡夫卡在生产者与主题之间设置了压缩类型 [英] Kafka set compression type at producer vs topic

问题描述

以下在kafka中启用压缩的方式有何区别?

What's the difference between the following ways of enabling compression in kafka:

方法1:使用以下命令创建主题:

Approach 1: Create a topic using the command:

bin/kafka-topics.sh --create --zookeeper localhost:2181 --config compression.type=gzip --topic test

方法2:在Kafka Producer客户端API中设置属性compression.type = gzip.

Approach 2: Set the property compression.type = gzip in Kafka Producer Client API.

使用方法1时，压缩效果更好，吞吐率更高.

I get better compression and higher throughput when using Approach 1.

如果使用方法1，是否意味着压缩发生在代理端，而在方法2中，消息在生产者端被压缩然后发送给代理?

If I use Approach 1, does it mean that the compression occurs at the broker end while in Approach 2, the messages are compressed at Producer end and then sent to broker?

推荐答案

如果我使用方法1，是否意味着压缩发生在代理端?

If I use Approach 1, does it mean that the compression occurs at the broker end?

这取决于.如果生产者未设置compression.type或设置其他值，则该消息将在代理端被压缩.但是，如果生产者还将compression.type设置为gzip，则无需在代理端再次进行压缩.实际上，还必须满足其他严格条件以确保无需压缩，尽管这超出了范围.

It depends. If the producer does not set a compression.type or sets a different one, then the message will be compressed at the broker end. However, if producer also sets compression.type to gzip, no need to compress again at the broker end. Actually, there are other strict conditions that must be met to ensure no need to compress, although it's a little bit beyond of the scope.

在方法2中，消息在生产者端被压缩，然后发送给代理?

in Approach 2, the messages are compressed at Producer end and then sent to broker?

是的，如果生产者设置其compression.type配置，则记录将被压缩，然后再发送给代理.

Yes, records will be compressed before being sent to the broker if producer sets its compression.type config.

这篇关于卡夫卡在生产者与主题之间设置了压缩类型的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！