如何使用python和boto3将Amazon S3文件下载到文件夹中的本地计算机上? [英] How to download Amazon S3 files on to local machine in folder using python and boto3?

问题描述

我正在尝试将文件从Amazon S3下载到本地计算机中的预定义文件夹中.这是代码，可以正常工作.但是，在保存文件时，它会以路径的姓氏保存.我该如何纠正?

I am trying to download a file from Amazon S3 to a predefined folder in the local machine. This is the code and it works fine. But when the file is saved, it saves with lastname of the path. How should I correct this?

import boto3
import os

S3_Object = boto3.client('s3', aws_access_key_id='##', aws_secret_access_key='##')
BUCKET_NAME = '##'
filename2 = []
Key2 = []
bucket = S3_Object.list_objects(Bucket=BUCKET_NAME)['Contents']
download_path = target_file_path = os.path.join('..', 'data', 'lz', 'test_sample', 'sample_file' )

for key in bucket:
    path, filename = os.path.split(key['Key'])
    filename2.append(filename)
    Key2.append(key['Key'])

for f in Key2:
    if f.endswith('.csv'):
        #if f.endswith('.csv'):
            print(f)           
            file_name = str(f.rsplit('/', 1)[-1])
            print(file_name)
            if not os.path.exists(download_path):
                os.makedirs(download_path)
            else:
                S3_Object.download_file(BUCKET_NAME, f, download_path + file_name)
                print("success")

推荐答案

这是我的测试代码.

import boto3
import os

s3 = boto3.resource('s3')
bucket = 'your bucket'
response = s3.Bucket(bucket).objects.all()
# If you want to search only specific path of bucket,
#response = s3.Bucket(bucket).objects.filter(Prefix='path')

path = 'your path'
if not os.path.exists(path):
    os.makedirs(path)

for item in response:
    filename = item.key.rsplit('/', 1)[-1]
    if filename.endswith('.csv'):
        s3.Object(bucket, item.key).download_file(path + filename)
        print("success")

我已经测试了代码，并给出了正确的名称.

I have tested the code and it gives a correct name.

怎么了?

我认为，您的代码中缺少该路径的/.

I think, there is a missing / in your code for the path.

print(os.path.join('..', 'data', 'lz', 'test_sample', 'sample_file'))

代码给出结果:

../data/lz/test_sample/sample_file

因此，在下面的步骤中，

So, in the below step,

S3_Object.download_file(BUCKET_NAME, f, download_path + file_name)

download_path + file_name将是错误的，应该是:

the download_path + file_name will be wrong and it should be:

S3_Object.download_file(BUCKET_NAME, f, download_path + '/' + file_name)

这篇关于如何使用python和boto3将Amazon S3文件下载到文件夹中的本地计算机上?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！