克服 S3 中的 1000 个存储桶限制/使用访问点 [英] Overcome 1000 bucket limit in S3 / use access points
问题描述
我每个客户有 1 个 s3 存储桶.客户是外部实体，他们不与任何其他人共享数据.我写入 S3，客户从 S3 读取.根据这个架构，我只能扩展到 1000 个存储桶，因为每个账户的 s3 存储桶是有限制的.我希望使用 AP 为每个客户创建 1 个 AP 并将数据放在一个桶中.然后客户可以使用 AP 从存储桶中读取文件.
I have 1 s3 bucket per customer. Customers are external entities and they dont share data with anyone else. I write to S3 and customer reads from S3. As per this architecture, I can only scale to 1000 buckets as there is a limit to s3 buckets per account. I was hoping to use APs to create 1 AP per customer and put data in one bucket. The customer can then read the files from the bucket using AP.
Bucket000001/prefix01.->客户帐户1
Bucket000001/prefix02.->customeraccount2...
S3 访问点要求您在访问点以及存储桶级别为 IAM 用户设置策略.如果我有 1000 个 IAM 用户，我是否需要为存储桶中的每个用户设置策略?这将导致一项巨大的政策.存储桶中有最大策略大小，所以我可能无法做到这一点.这是接入点可以提供帮助的正确用例吗?
S3 access points require you to set policy for a IAM user in access point as well as the bucket level. If I have 1000s of IAM users, do I need to set policy for each of them in the bucket? This would result in one giant policy. there is a max policy size in the bucket, so I may not be able to do that. Is this the right use case where access points can help?
推荐答案
推荐的方法是:
- 不要将 IAM 用户分配给您的客户.这些类型的 AWS 凭证只能由您的内部员工和您自己的应用程序使用.
- 您应该提供一个 Web 应用程序(或 API)，客户可以在其中根据您自己的用户数据库进行身份验证(或者您可以使用 Amazon Cognito 来管理身份验证).
- 通过身份验证后，应用程序应授予对Web 界面的访问权限以访问 Amazon S3，或者应用程序应提供临时凭证以访问 Amazon S3(更多详细信息如下).
- 不要为每个客户使用一个存储桶.这是不可扩展的.相反，将所有客户数据存储在一个存储桶中，每个用户都有自己的文件夹.您可以在 Amazon S3 中存储的数据量没有限制.这也使您更容易管理和维护，因为跨所有内容执行功能更容易，而不必进入单独的存储桶.(如果您希望按客户位置(地区)或客户类型对存储分区进行细分，则可能是一个例外.但不要为每个客户使用一个存储分区.没有理由这样做.)
- 在授予对 Amazon S3 的访问权限时，在文件夹级别分配权限以确保客户只能看到自己的数据.
- Do NOT assign IAM Users to your customers. These types of AWS credentials should only be used by your internal staff and your own applications.
- You should provide a web application (or an API) where customers can authenticate against your own user database (or you could use Amazon Cognito to manage authentication).
- Once authenticated, the application should grant access either to a web interface to access Amazon S3, or the application should provide temporary credentials for accessing Amazon S3 (more details below).
- Do not use one bucket per customer. This is not scalable. Instead, store all customer data in ONE bucket, with each user having their own folder. There is no limit on the amount of data you can store in Amazon S3. This also makes it easier for you to manage and maintain, since it is easier to perform functions across all content rather than having to go into separate buckets. (An exception might be if you wish to segment buckets by customer location (region) or customer type. But do not use one bucket per customer. There is no reason to do this.)
- When granting access to Amazon S3, assign permissions at the folder-level to ensure customers only see their own data.
选项 1:通过 Web 应用程序访问
如果您的客户通过 Web 应用程序访问 Amazon S3，那么您可以对该应用程序进行编码以在文件夹级别实施安全性.例如，当他们请求文件列表时，只显示其文件夹内的文件.
If your customers access Amazon S3 via a web application, then you can code that application to enforce security at the folder level. For example, when they request a list of files, only display files within their folder.
这种安全性可以完全在您自己的代码中进行管理.
This security can be managed totally within your own code.
选项 2:通过临时凭证访问
如果您的客户使用编程访问(例如使用 AWS CLI 或在其系统上运行的自定义应用程序)，则:
If your customers use programmatic access (eg using the AWS CLI or a custom app running on their systems), then:
- 客户应该对您的应用程序进行身份验证(如何进行身份验证取决于您对用户进行身份验证的方式)
- 一旦通过身份验证，应用程序应使用生成临时凭证AWS 安全令牌服务 (STS).在生成凭证时，授予对 Amazon S3 的访问权限，但在 ARN 中指定客户的文件夹(例如
arn:aws:s3:::storage-bucket/customer1/*)，以便他们只能访问内容在他们的文件夹中. - 将这些临时凭据返回给客户.然后，他们可以使用这些凭证直接对 Amazon S3 进行 API 调用(例如从 AWS 命令行界面 (CLI) 或自定义应用程序).他们将被限制在自己的文件夹中.
- The customer should authenticate to your application (how this is done will vary depending upon how you are authenticating users)
- Once authenticated, the application should generate temporary credentials using the AWS Security Token Service (STS). While generating the credentials, grant access to Amazon S3 but specify the customer's folder in the ARN (eg
arn:aws:s3:::storage-bucket/customer1/*) so that they can only access content within their folder. - Return these temporary credentials to the customer. They can then use these credentials to make API calls directly to Amazon S3 (eg from the AWS Command-Line Interface (CLI) or a custom app). They will be limited to their own folder.
这种方法通常用于移动应用.移动应用程序针对后端进行身份验证，接收临时凭证，然后使用这些凭证直接与 S3 交互.因此，后端应用程序仅用于身份验证.
This approach is commonly done with mobile applications. The mobile app authenticates against the backend, receives temporary credentials, then uses those credentials to interact directly against S3. Thus, the back-end app is only used for authentication.
YouTube 上的示例:
Examples on YouTube:
这篇关于克服 S3 中的 1000 个存储桶限制/使用访问点的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
