存储桶之间的AWS S3同步将覆盖较新的目标文件 [英] AWS S3 sync between buckets overwriting newer destination files

问题描述

我们有两个s3存储桶，并且有一个同步cron作业，应将bucket1的更改复制到bucket2.

We have two s3 buckets, and we have a sync cron job that should copy bucket1 changes to bucket2.

aws s3 sync s3://bucket1/images/ s3://bucket2/images/

将新图像添加到存储区1时，它会正确地复制到存储区2中.

When a new image is added to bucket1, it correctly gets copied over to bucket2.

但是，如果我们将该图像的新版本上载到bucket2，则在下次运行同步作业时，它实际上会将旧版本从bucket1复制到bucket2，从而替换了我们刚才放置的较新版本.

However, if we upload a new version of that image to bucket2, when the sync job next runs it actually copies the older version from bucket1 over to bucket2, replacing the newer version we just put there.

这是迁移过程的一部分，及时将图像上传到的唯一位置是bucket2，但暂时有时它们可​​以上传至其中一个，并且我们只希望复制来自bucket1的更改到bucket2，而不是相反.

This is part of a migration process, and in time the only place images will be uploaded to will be bucket2, but for the time being sometimes they may be uploaded to either, and we only want changes form bucket1 to be copied up to bucket2, NOT the other way round.

为什么aws sync工作似乎认为bucket1上的文件已更改?是否不知道bucket2中的文件是较新的文件，因此应该将其保留?

Why does the aws sync job seem to think that the file on bucket1 has changed? Does it not know that the file in bucket2 is newer, so it should be left alone?

推荐答案

AWS命令行界面(CLI) aws s3 sync命令将内容从源"位置复制到目标"位置.它只会复制自上次同步以来已添加或更改的文件.

The AWS Command-Line Interface (CLI) aws s3 sync command copies content from the Source location to the Destination location. It only copies files that have been added or changed since the last sync.

它被设计为单向同步，而不是双向同步.您的文件被覆盖，因为源中的文件不存在于目标中.这是正确的行为.

It is designed as a one-way sync, not a two-way sync. Your file is being overwritten because the file in the Source is not present in the Destination. This is correct behavior.

调整这些控件的范围非常有限，例如(来自

There is limited range to tweak these controls, such as (from the sync command documentation):

--exact-timestamps(布尔值)从S3同步到本地时，只有当时间戳完全匹配时，相同大小的项目才会被忽略.默认行为是忽略大小相同的项目，除非本地版本比S3版本新.

--exact-timestamps (boolean) When syncing from S3 to local, same-sized items will be ignored only when the timestamps match exactly. The default behavior is to ignore same-sized items unless the local version is newer than the S3 version.

但是，似乎没有一个选项可以仅由于存在同名文件或具有保留较新文件的首选项而停止覆盖文件的选项.

However, there does not appear to be an option that stops overwriting of files merely because a file with the same name exists, or something with a preference to keep newer files.

如果您想使用更具体的规则进行双向同步，则需要自己编写代码.

If you want a two-way sync with more specific rules, you will need to code it yourself.

这篇关于存储桶之间的AWS S3同步将覆盖较新的目标文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！