S3 SlowDown:请减少您的请求率异常 [英] S3 SlowDown: Please reduce your request rate exception

问题描述

我有一个服务，该服务每秒钟向S3发送10k个PUT请求. S3能够处理几分钟的负载，但此后开始引发SlowDown异常.这使我的服务减慢到无法接受的速度.

I have a service that sends 10k PUT requests to S3 for every second. The S3 was able to handle those loads for several minutes but started to throw SlowDown exception after that. It slows down my service to an unacceptable rate.

我已阅读此并实施了建议的最佳做法.以下是前缀的格式:bucket-name/[First four of UUID]-[YYYYmmddhhiiss]/[random UUID]/[random UUID].json.但是该方法不起作用.

I have read this and implemented the suggested best practice. Following is the format of the prefix: bucket-name/[First four of UUID]-[YYYYmmddhhiiss]/[random UUID]/[random UUID].json. The method didn't work though.

任何想法如何克服此错误?谢谢！ 附注:我已要求提高AWS支持中心的PUT限额.他们建议上述步骤无效.

Any idea how to overcome this error? Thank you! P.S: I have requested the PUT limit increase to AWS Support center. They suggested the above steps which didn't work.

推荐答案

S3已分发，您需要确保不创建热点.您可以通过确保对象键是真正随机的来避免这种情况.

S3 is distributed, and you need to make sure that you don't create hotspots. You can avoid this by ensuring that your object keys are truly random.

因此将[randome UUID]移到对象密钥的第一部分.而且，如果您没有生成真正的随机UUID，(例如，听起来每个对象的前4个字符都相似)，请尝试反转UUID.

So move the [randome UUID] to the first part of your object key. And if you're not generating truly random UUIDs, (i.e. it sounds like perhaps the first 4 characters are similar for each object) try reversing the UUID.

可以找到更多提示 查看全文