Kafka:Confluent 的 HDFS 连接器是否支持 Snappy 压缩? [英] Kafka: Does Confluent’s HDFS connector support Snappy compression?

问题描述

我在 HDFS 连接器文档中没有看到任何压缩配置 https://docs.confluent.io/current/connect/connect-hdfs/docs/configuration_options.html.

I don't see any configurations for compression on the HDFS connector docs https://docs.confluent.io/current/connect/connect-hdfs/docs/configuration_options.html.

• 是否支持压缩?
• 如果是，我需要在属性文件中添加什么?

推荐答案

Snappy 压缩最近被添加到 HDFS Connector for Avro.要启用它，您需要将属性 avro.codec 设置为 snappy.

Snappy compression was recently added to the HDFS Connector for Avro. To enable it you'll need to set the property avro.codec to snappy.

Parquet 从一开始就可用，它是导出 parquet 文件时使用的编解码器.

With Parquet it's been available since the beginning and it is the codec used when exporting parquet files.

这是为 Avro 添加的 PR:

Here's the PR that added it for Avro:

https://github.com/confluentinc/kafka-connect-hdfs/pull/255

此配置属性将在即将发布的连接器 4.1.0 版本中可用.这也是文档链接将更新以反映此添加的时间.

This config property will become available in the forthcoming 4.1.0 release of the connector. That's also when the link to the docs will be updated to reflect this addition.

这篇关于Kafka:Confluent 的 HDFS 连接器是否支持 Snappy 压缩?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！