如何查找 S3 存储桶中文件夹的大小? [英] How to find size of a folder inside an S3 bucket?

问题描述

我在 python 中使用 boto3 模块与 S3 交互，目前我能够获取 S3 存储桶中每个单独键的大小.但我的动机是只找到顶级文件夹的空间存储(每个文件夹都是不同的项目)，我们需要根据使用的空间对每个项目收费.我能够获得顶级文件夹的名称，但无法获得有关以下实现中文件夹大小的任何详细信息.以下是我获取顶级文件夹名称的实现.

I am using boto3 module in python to interact with S3 and currently I'm able to get the size of every individual key in an S3 bucket. But my motive is to find the space storage of only the top level folders (every folder is a different project) and we need to charge per project for the space used. I'm able to get the names of the top level folders but not getting any details about the size of the folders in the below implementation. The following is my implementation to get the top level folder names.

import boto
import boto.s3.connection

AWS_ACCESS_KEY_ID = "access_id"
AWS_SECRET_ACCESS_KEY = "secret_access_key"
Bucketname = 'Bucket-name' 

conn = boto.s3.connect_to_region('ap-south-1',
   aws_access_key_id=AWS_ACCESS_KEY_ID,
   aws_secret_access_key=AWS_SECRET_ACCESS_KEY,
   is_secure=True, # uncomment if you are not using ssl
   calling_format = boto.s3.connection.OrdinaryCallingFormat(),
   )

bucket = conn.get_bucket('bucket')
folders = bucket.list("", "/")

for folder in folders:
    print(folder.name)

这里的文件夹类型是 boto.s3.prefix.Prefix，它不显示任何大小的细节.有没有办法通过名称搜索 S3 存储桶中的文件夹/对象，然后获取该对象的大小?

The type of folder here is boto.s3.prefix.Prefix and it doesn't display any details of size. Is there any way to search a folder/object in an S3 bucket by it's name and then fetch the size of that object ?

推荐答案

在S3中查找顶级文件夹"的大小(S3没有真的有文件夹的概念，但是有点在用户界面中显示文件夹结构)，这样的事情会起作用:

To find the size of the top-level "folders" in S3 (S3 does not really have a concept of folders, but kind of displays a folder structure in the UI), something like this will work:

from boto3 import client
conn = client('s3')

top_level_folders = dict()

for key in conn.list_objects(Bucket='kitsune-buildtest-production')['Contents']:

    folder = key['Key'].split('/')[0]
    print("Key %s in folder %s. %d bytes" % (key['Key'], folder, key['Size']))

    if folder in top_level_folders:
        top_level_folders[folder] += key['Size']
    else:
        top_level_folders[folder] = key['Size']


for folder, size in top_level_folders.items():
    print("Folder: %s, size: %d" % (folder, size))

这篇关于如何查找 S3 存储桶中文件夹的大小?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！