使用 Apache Airflow 将文件从一个 Google Cloud Storage Bucket 复制到另一个 [英] Copy files from one Google Cloud Storage Bucket to other using Apache Airflow

问题描述

问题:我想将文件从 Google Cloud Storage Bucket 中的文件夹(例如 Bucket1 中的 Folder1)复制到另一个 Bucket(例如 Bucket2).我找不到任何用于 Google Cloud Storage 复制文件的 Airflow Operator.

Problem: I want to copy files from a folder in Google Cloud Storage Bucket (e.g Folder1 in Bucket1) to another Bucket (e.g Bucket2). I can't find any Airflow Operator for Google Cloud Storage to copy files.

推荐答案

我刚刚在2小时前上传的contrib中发现了一个新的算子:https://github.com/apache/incubator-airflow/blob/master/airflow/contrib/operators/gcs_to_gcs.py 调用 GoogleCloudStorageToGoogleCloudStorageOperator，它应该将一个对象从一个存储桶复制到另一个存储桶，并根据要求重命名.

I just found a new operator in contrib uploaded 2 hours ago: https://github.com/apache/incubator-airflow/blob/master/airflow/contrib/operators/gcs_to_gcs.py called GoogleCloudStorageToGoogleCloudStorageOperator that should copy an object from a bucket to another, with renaming if requested.

这篇关于使用 Apache Airflow 将文件从一个 Google Cloud Storage Bucket 复制到另一个的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！