带进度条的 S3 Python 下载 [英] S3 Python Download with Progress Bar

问题描述

无法评论我修改此代码的初始线程(使用 boto3 和回调跟踪 S3 文件的下载进度) 所以希望有人能在这里帮助我.我能够使用此代码来显示文件上传的进度条，现在我需要为从 AWS S3 下载文件执行相同的操作.任何帮助将不胜感激！

Couldn't comment on the initial thread where I adapted this code (Track download progress of S3 file using boto3 and callbacks) so hopefully someone can help me here. I was able to use this code to show a progress bar for file uploads, now I need the do the same thing for file downloads from AWS S3. Any help would be GREATLY APPRECIATED!

我知道我需要从 S3 而不是从本地文件系统获取文件的大小.我确定我需要调整一些愚蠢的代码才能使其工作.希望有人能有所启发.:)

I know I need to get the size of the file from S3 instead of from the local file system. I'm sure there is some silly code I need to adjust to make this work. Hopefully, someone can shed some light. :)

def upload_to_aws(local_file, bucket, s3_file):
    s3 = boto3.client('s3', aws_access_key_id=s3ak,
                      aws_secret_access_key=s3sk)

    statinfo = os.stat(local_file)
    up_progress = progressbar.progressbar.ProgressBar(maxval=statinfo.st_size)
    up_progress.start()
    def upload_progress(chunk):
        up_progress.update(up_progress.currval + chunk)

    try:
        s3.upload_file(local_file, bucket, s3_file, Callback=upload_progress)
        up_progress.finish()
        print("Upload Successful")
        return True
    except FileNotFoundError:
        print("The file was not found")
        return False
    except NoCredentialsError:
        print("Credentials not available")
        return False

推荐答案

import os
import boto3
import progressbar
from botocore.exceptions import NoCredentialsError


def download_from_s3(bucket, s3_file, local_file):
    s3 = boto3.client('s3')
    response = s3.head_object(Bucket=bucket, Key=s3_file)
    size = response['ContentLength']
    up_progress = progressbar.progressbar.ProgressBar(maxval=size)
    up_progress.start()

    def upload_progress(chunk):
        up_progress.update(up_progress.currval + chunk)

    try:
        s3.download_file(bucket, s3_file, local_file, 
Callback=upload_progress)
        up_progress.finish()
        print("Download Successful")
        return True
    except FileNotFoundError:
        print("The file was not found")
        return False
    except NoCredentialsError:
        print("Credentials not available")
        return False


resp = download_from_s3(bucket, s3_file, local_file)

这篇关于带进度条的 S3 Python 下载的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！