Kafka：Confluent的HDFS连接器是否支持Snappy压缩？ [英] Kafka: Does Confluent’s HDFS connector support Snappy compression?

问题描述

我没有在HDFS连接器上看到任何用于压缩的配置文件docs https://docs.confluent.io/current/connect/connect-hdfs/docs/configuration_options.html 。

I don't see any configurations for compression on the HDFS connector docs https://docs.confluent.io/current/connect/connect-hdfs/docs/configuration_options.html.

• 它支持压缩吗？


• 如果是，我需要在属性文件中添加什么？

推荐答案

Snappy压缩最近添加到了用于Avro的HDFS连接器中。要启用它，您需要将属性 avro.codec 设置为 snappy 。

Snappy compression was recently added to the HDFS Connector for Avro. To enable it you'll need to set the property avro.codec to snappy.

Parquet自一开始就可用，它是导出Parquet文件时使用的编解码器。

With Parquet it's been available since the beginning and it is the codec used when exporting parquet files.

以下是为Avro添加它的PR：

Here's the PR that added it for Avro:

> https://github.com/confluentinc/kafka-connect-hdfs/pull/255

此配置属性将在即将发布的连接器4.1.0版本中提供。这也是指向文档链接的更新，以反映此添加。

This config property will become available in the forthcoming 4.1.0 release of the connector. That's also when the link to the docs will be updated to reflect this addition.

这篇关于Kafka：Confluent的HDFS连接器是否支持Snappy压缩？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！