如何在S3存储桶中查找文件夹的大小? [英] How to find size of a folder inside an S3 bucket?

问题描述

我正在python中使用boto3模块与S3进行交互，目前我能够获取S3存储桶中每个键的大小.但是我的动机是找到仅顶层文件夹的空间存储(每个文件夹都是一个不同的项目)，我们需要为每个项目收取使用空间的费用.我能够获取顶级文件夹的名称，但在以下实现中无法获取有关文件夹大小的任何详细信息.以下是我获取顶级文件夹名称的实现.

I am using boto3 module in python to interact with S3 and currently I'm able to get the size of every individual key in an S3 bucket. But my motive is to find the space storage of only the top level folders (every folder is a different project) and we need to charge per project for the space used. I'm able to get the names of the top level folders but not getting any details about the size of the folders in the below implementation. The following is my implementation to get the top level folder names.

import boto
import boto.s3.connection

AWS_ACCESS_KEY_ID = "access_id"
AWS_SECRET_ACCESS_KEY = "secret_access_key"
Bucketname = 'Bucket-name' 

conn = boto.s3.connect_to_region('ap-south-1',
   aws_access_key_id=AWS_ACCESS_KEY_ID,
   aws_secret_access_key=AWS_SECRET_ACCESS_KEY,
   is_secure=True, # uncomment if you are not using ssl
   calling_format = boto.s3.connection.OrdinaryCallingFormat(),
   )

bucket = conn.get_bucket('bucket')
folders = bucket.list("", "/")

for folder in folders:
    print(folder.name)

这里的文件夹类型是boto.s3.prefix.Prefix，它不显示任何大小详细信息.是否可以通过名称搜索S3存储桶中的文件夹/对象，然后获取该对象的大小?

The type of folder here is boto.s3.prefix.Prefix and it doesn't display any details of size. Is there any way to search a folder/object in an S3 bucket by it's name and then fetch the size of that object ?

推荐答案

要在S3中查找顶级文件夹"的大小(S3并不是确实有文件夹的概念，但是在UI中显示一种文件夹结构)，类似这样的方法将起作用:

To find the size of the top-level "folders" in S3 (S3 does not really have a concept of folders, but kind of displays a folder structure in the UI), something like this will work:

from boto3 import client
conn = client('s3')

top_level_folders = dict()

for key in conn.list_objects(Bucket='kitsune-buildtest-production')['Contents']:

    folder = key['Key'].split('/')[0]
    print("Key %s in folder %s. %d bytes" % (key['Key'], folder, key['Size']))

    if folder in top_level_folders:
        top_level_folders[folder] += key['Size']
    else:
        top_level_folders[folder] = key['Size']


for folder, size in top_level_folders.items():
    print("Folder: %s, size: %d" % (folder, size))

这篇关于如何在S3存储桶中查找文件夹的大小?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！