如何使用Google Cloud Dataflow将压缩文件写入Google Cloud Storage? [英] How do I write compressed files to Google Cloud Storage using Google Cloud Dataflow?

问题描述

我正在尝试在Google Dataflow程序中将Gzip压缩文件写入Google Cloud Storage存储桶.常见问题解答说

I am trying to write Gzipped files into Google Cloud Storage buckets in a Google Dataflow program. The FAQs say that

Does the TextIO source and sink support compressed files, such as GZip?
Yes. Cloud Dataflow can read files compressed with gzip and bzip2.

这是否意味着它们不支持编写GZip文件?

Does this mean that they don't support writing of GZip files?

推荐答案

正确，我们目前没有内置的支持来编写gzip文件. 但是，用户定义的数据格式API 在特定的 FileBasedSink ，应该可以很容易地自己编写一个接收器.

Correct, we currently don't have built-in support for writing gzip files. However, the user-defined data format API, in particular FileBasedSink, should make it straightforward to write a sink like that yourself.

这篇关于如何使用Google Cloud Dataflow将压缩文件写入Google Cloud Storage?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！