如何使用AWS Data Pipeline将文件从一个S3存储桶/目录传输到另一个 [英] How to transfer a file/files from one S3 bucket/directory to another using AWS Data Pipeline

问题描述

我想使用AWS数据管道将文件从一个S3存储桶目录传输到另一个目录(即，将其复制到目标目录并从源目录中删除).

I would like to transfer a file (i.e copy it to a target directory and delete it from the source directory) from one S3 bucket directory to another using AWS data pipeline.

我尝试使用ShellCommandActivity并制作了一个脚本，该脚本可以将文件从一个S3存储桶/目录移动到另一个.但是结果是它仅将其复制到目标S3存储桶/目录，而没有删除S3源目录中的文件.

I tried using the ShellCommandActivity and made a script that would move a file/files from one S3 bucket/directory to another. But the result was that it only copied it to the target S3 bucket/directory and did not remove the file in the S3 source directory.

提前谢谢！

推荐答案

如果您要从S3存储桶中删除某些内容，则必须明确删除它；除了复制和删除之外，没有移动"的本机概念.

If you want to remove something from an S3 bucket you must explicitly delete it; there is no native concept of "move" other than copy and delete.

也就是说，上次尝试时，我发现无法从数据管道中的S3存储桶资源中删除.我不得不通过使用s3cmd来解决此问题.

That said, last time I tried I found I could not delete from an S3 bucket resource in the data pipeline. I had to get around this by using s3cmd instead.

这篇关于如何使用AWS Data Pipeline将文件从一个S3存储桶/目录传输到另一个的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！