Boto3 下载文件中的 IOError [英] IOError in Boto3 download_file

问题描述

背景

我正在使用以下 Boto3 代码从 S3 下载文件.

I am using the following Boto3 code to download file from S3.

for record in event['Records']:
    bucket = record['s3']['bucket']['name']
    key = record['s3']['object']['key']
    print (key)
    if key.find('/') < 0 :
    if len(key) > 4 and key[-5:].lower() == '.json': //File is uploaded outside any folder

        download_path = '/tmp/{}{}'.format(uuid.uuid4(), key)
    else:
        download_path = '/tmp/{}/{}'.format(uuid.uuid4(), key)//File is uploaded inside a folder

如果在 S3 存储桶中上传了新文件，则会触发此代码并通过此代码下载新上传的文件.

If a new file is uploaded in S3 bucket, this code is triggered and that newly uploaded file is downloaded by this code.

当上传到任何文件夹之外时，此代码工作正常.

This code works fine when uploaded outside any folder.

但是，当我在目录中上传文件时，会发生 IO 错误.这是我遇到的 IO 错误的转储.

However, when I upload a file inside a directory, IO error happens. Here is a dump of the IO error I am encountering.

[Errno 2] 没有这样的文件或目录:/tmp/316bbe85-fa21-463b-b965-9c12b0327f5d/test1/customer1.json.586ea9b8:IO错误

[Errno 2] No such file or directory: /tmp/316bbe85-fa21-463b-b965-9c12b0327f5d/test1/customer1.json.586ea9b8: IOError

test1 是我的 S3 存储桶中上传 customer1.json 的目录.

test1 is the directory inside my S3 bucket where customer1.json is uploaded.

查询

有关如何解决此错误的任何想法?

Any thoughts on how to resolve this error?

推荐答案

由于您尝试下载文件并将文件保存到不存在的目录中而引发错误.在下载文件之前使用 os.mkdir 创建目录.

Error raised because you attempted to download and save file into directory which not exists. Use os.mkdir prior downloading file to create an directory.

# ...
else:
    item_uuid = str(uuid.uuid4())
    os.mkdir('/tmp/{}'.format(item_uuid))
    download_path = '/tmp/{}/{}'.format(item_uuid, key)  # File is uploaded inside a folder

注意:最好使用 os.path.join() 使用系统路径操作时.所以上面的代码可以重写为:

Note: It's better to use os.path.join() while operating with systems paths. So code above could be rewritten to:

# ...
else:
    item_uuid = str(uuid.uuid4())
    os.mkdir(os.path.join(['tmp', item_uuid]))
    download_path = os.path.join(['tmp', item_uuid, key]))

也可能会引发错误，因为您在 s3 存储桶文件的下载路径中包含/tmp/"，不包含 tmp 文件夹，因为它可能不存在于 s3 上.使用这些文章确保您走在正确的道路上:

Also error may be raises because you including '/tmp/' in download path for s3 bucket file, do not include tmp folder as likely it's not exists on s3. Ensure you are on the right way by using that articles:

• Amazon S3使用 Python/Django 上传和下载

Python s3 示例

这篇关于Boto3 下载文件中的 IOError的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！