使用Apache Airflow将文件从一个Google Cloud Storage Bucket复制到另一个文件 [英] Copy files from one Google Cloud Storage Bucket to other using Apache Airflow

问题描述

问题：我想将文件从Google Cloud Storage Bucket中的一个文件夹（例如Bucket1中的Folder1）复制到另一个Bucket（例如Bucket2）。我找不到用于Google Cloud Storage的任何Airflow操作员来复制文件。

Problem: I want to copy files from a folder in Google Cloud Storage Bucket (e.g Folder1 in Bucket1) to another Bucket (e.g Bucket2). I can't find any Airflow Operator for Google Cloud Storage to copy files.

推荐答案

我刚刚在contrib中找到了一个新操作员2小时前： https://github.com名为 GoogleCloudStorageToGoogleCloudStorageOperator 的com / apache / incubator-airflow / blob / master / airflow / contrib / operators / gcs_to_gcs.py 将该对象从存储桶复制到另一个存储桶，并根据需要重命名。

I just found a new operator in contrib uploaded 2 hours ago: https://github.com/apache/incubator-airflow/blob/master/airflow/contrib/operators/gcs_to_gcs.py called GoogleCloudStorageToGoogleCloudStorageOperator that should copy an object from a bucket to another, with renaming if requested.

这篇关于使用Apache Airflow将文件从一个Google Cloud Storage Bucket复制到另一个文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！