Python Boto3-如何在进程开始复制到另一个存储桶之前检查s3文件是否已完全写入 [英] Python Boto3 - how to check if s3 file is completely written before process start copying to another bucket

问题描述

在流程B开始使用boto3将文件复制到AWS S3存储桶B之前，如何确保流程A在AWS S3存储桶A中完全写入了大文件(5 GB以上)?

How to make sure that Process A has completely written large file (5+ GB) in AWS S3 Bucket A before Process B starts copying file to AWS S3 Bucket B using boto3?

推荐答案

如果在Amazon S3中创建了新对象，则仅在上传完成后才会出现.上传完成之前，其他进程将无法查看它.

If a new object is being created in Amazon S3, it will only appear after the upload is complete. Other processes will not be able to view it until is has finished uploading.

在S3中不能更新对象.而是将它们替换为新对象.因此，如果某个对象正在更新过程中，它将仍然作为其他过程中的旧对象出现.

Objects cannot be updated in S3. Rather, they are replaced with a new object. So, if an object is in the process of being updated, it will still appear as the old object to other processes.

最好的方法是通过配置Amazon S3来触发流程B活动通知.上传新对象后，S3可以触发Lambda函数(或发送通知)，然后执行第二步.

The best way would be to trigger Process B by Configuring Amazon S3 Event Notifications. Once the new object is uploaded, S3 can trigger a Lambda function (or send a notification) that can then perform the second step.

这篇关于Python Boto3-如何在进程开始复制到另一个存储桶之前检查s3文件是否已完全写入的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！