有多个 s3 存储桶还是一个带有子文件夹的存储桶更好? [英] Is it better to have multiple s3 buckets or one bucket with sub folders?

问题描述

每个上传类别有多个 s3 存储桶，还是一个带有子文件夹的存储桶或链接的 s3 存储桶更好?我确信用户图片的数量会多于个人图片的数量，并且每个存储桶限制为 5TB，每个帐户限制为 100 个存储桶.我正在使用 aws boto 库和 https://github.com/amol-/depot

Is it better to have multiple s3 buckets per category of uploads or one bucket with sub folders OR a linked s3 bucket? I know for sure there will be more user-images than there will be profille-pics and that there is a 5TB limit per bucket and 100 buckets per account. I'm doing this using aws boto library and https://github.com/amol-/depot

我的文件夹结构采用以下哪种方式?

Which is the structure my folders in which of the following manner?

/app_bucket
    /profile-pic-folder
    /user-images-folder

OR

profile-pic-bucket
user-images-bucket


OR


/app_bucket_1
/app_bucket_2

最后一个意味着它实际上是一个 10TB 存储桶，其中当存储桶_1 中的文件超过 5TB 时创建一个新存储桶.但是所有上传的内容都会像在一个存储桶中一样被读取.或者有没有更好的方法来做我想做的事情?非常感谢！

The last one implies that its really a 10TB bucket where a new bucket is created when the files within bucket_1 exceeds 5TB. But all uploads will be read as if in one bucket. Or is there a better way of doing what I'm trying to do? Many thanks!

我不确定这是否正确...每个帐户 100 个存储桶?

I'm not sure if this is correct... 100 buckets per account?

https://www.reddit.com/r/aws/comments/28vbjs/requesting_increase_in_number_of_s3_buckets/

推荐答案

是的，每个帐户实际上有 100 个存储桶的限制.我在 AWS 活动中向架构师询问了原因.他说这是为了避免人们在 S3 上托管无限的静态网站，因为他们认为这可能会被滥用.但是你可以申请加薪.

Yes, there is actually a 100 bucket limit per account. I asked the reason for that to an architect in an AWS event. He said this is to avoid people hosting unlimited static websites on S3 as they think this may be abused. But you can apply for an increase.

默认情况下，您最多可以在每个 AWS 中创建 100 个存储桶帐户.如果你需要额外的桶，你可以增加你的桶通过提交服务限制增加来限制.

By default, you can create up to 100 buckets in each of your AWS accounts. If you need additional buckets, you can increase your bucket limit by submitting a service limit increase.

来源:http://docs.aws.amazon.com/AmazonS3/latest/dev/BucketRestrictions.html

另外，请注意，在 S3 中实际上没有文件夹，只是一个平面文件结构:

Also, please note that there are actually no folders in S3, just a flat file structure:

Amazon S3 具有扁平结构，没有您在 中看到的层次结构一个典型的文件系统.但是，为了组织简单，Amazon S3 控制台支持文件夹概念作为对象分组的方法.Amazon S3 通过使用密钥名称来做到这一点对象的前缀.

Amazon S3 has a flat structure with no hierarchy like you would see in a typical file system. However, for the sake of organizational simplicity, the Amazon S3 console supports the folder concept as a means of grouping objects. Amazon S3 does this by using key name prefixes for objects.

来源:http://docs.aws.amazon.com/AmazonS3/latest/UG/FolderOperations.html

最后，5TB 限制仅适用于单个对象.对象数量或桶的总大小没有限制.

Finally, the 5TB limit only applies to a single object. There is no limit on the number of objects or total size of the bucket.

问:我可以存储多少数据?

Q: How much data can I store?

您可以存储的数据总量和对象数量是无限制.

The total volume of data and number of objects you can store are unlimited.

来源:https://aws.amazon.com/s3/faqs/

文档还指出，使用单个存储桶或多个存储桶之间没有性能差异，因此我想选项 1 和 2 都适合您.

Also the documentation states there is no performance difference between using a single bucket or multiple buckets so I guess both option 1 and 2 would be suitable for you.

希望这会有所帮助.

这篇关于有多个 s3 存储桶还是一个带有子文件夹的存储桶更好?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！