当超过预配置的吞吐量时应该怎么做? [英] What should be done when the provisioned throughput is exceeded?

问题描述

我使用 AWS SDK for Javascript (Node.js) 从 DynamoDB 表中读取数据.Auto Scaling 功能在大部分时间都表现出色，并且一天中大部分时间消耗的读取容量单位 (RCU) 非常低.但是，有一个在午夜前后执行的编程作业，它消耗的 RCU 大约是预配 RCU 的 10 倍，并且由于自动缩放需要一些时间来调整容量，因此存在大量受限制的读取请求.此外，我怀疑我的请求没有完成(尽管我在错误日志中找不到任何异常).

I'm using AWS SDK for Javascript (Node.js) to read data from a DynamoDB table. The auto scaling feature does a great job during most of the time and the consumed Read Capacity Units (RCU) are really low most part of the day. However, there's a programmed job that is executed around midnight which consumes about 10x the provisioned RCU and since the auto scaling takes some time to adjust the capacity, there are a lot of throttled read requests. Furthermore, I suspect my requests are not being completed (though I can't find any exceptions in my error log).

为了处理这种情况，我考虑使用 AWS API 增加预置的 RCU (updateTable) 但计算我的应用程序需要的 RCU 数量可能并不简单.

In order to handle this situation, I've considered increasing the provisioned RCU using the AWS API (updateTable) but calculating the number of RCU my application needs may not be straightforward.

所以我的第二个猜测是重试失败的请求，然后等待自动缩放增加预配的 RCU.正如 AWS 文档和一些 Stack Overflow 的回答所指出的(特别是关于 ProvisionedThroughputExceededException):

So my second guess was to retry failed requests and simply wait for auto scale increase the provisioned RCU. As pointed out by AWS docs and some Stack Overflow answers (particularlly about ProvisionedThroughputExceededException):

适用于 Amazon DynamoDB 的 AWS 开发工具包会自动重试收到此异常的请求.因此，您的请求最终会成功，除非请求太大或您的重试队列太大而无法完成.

The AWS SDKs for Amazon DynamoDB automatically retry requests that receive this exception. So, your request is eventually successful, unless the request is too large or your retry queue is too large to finish.

我读过类似的问题(这个，这个和这个)但我仍然很困惑:如果请求太大或重试队列太大而无法完成(因此在自动重试之后)还是实际上在重试之前?

I've read similar questions (this one, this one and this one) but I'm still confused: is this exception raised if the request is too large or the retry queue is too large to finish (therefore after the automatic retries) or actually before the retries?

最重要的是:在我的上下文中我应该期待例外吗?(所以我可以抓住它并重试直到自动缩放增加 RCU?)

Most important: is that the exception I should be expecting in my context? (so I can catch it and retry until auto scale increases the RCU?)

推荐答案

是.

每次您的应用程序发送超出您容量的请求时，您都会从 Dynamo 收到 ProvisionedThroughputExceededException 消息.但是，您的 SDK 会为您处理并重试.Dynamo 默认重试时间从 50ms 开始，默认重试次数为 10，默认为指数退避.

Every time your application sends a request that exceeds your capacity you get ProvisionedThroughputExceededException message from Dynamo. However your SDK handles this for you and retries. The default Dynamo retry time starts at 50ms, the default number of retries is 10, and backoff is exponential by default.

这意味着您可以在以下位置重试:

This means you get retries at:

• 50 毫秒
• 100 毫秒
• 200 毫秒
• 400 毫秒
• 800 毫秒
• 1.6 秒
• 3.2 秒
• 6.4 秒
• 12.8 秒
• 25.6 秒

如果在第 10 次重试后您的请求仍未成功，SDK 会将 ProvisionedThroughputExceededException 传递回您的应用程序，您可以按照自己的意愿处理它.

If after the 10th retry your request has still not succeeded, the SDK passes the ProvisionedThroughputExceededException back to your application and you can handle it how you like.

您可以通过增加吞吐量供应来处理它，但另一种选择是在创建 Dynamo 连接时更改默认重试次数.例如

You could handle it by increasing throughput provision but another option would be to change the default retry times when you create the Dynamo connection. For example

new AWS.DynamoDB({maxRetries: 13, retryDelayOptions: {base: 200}});

这意味着您重试 13 次，初始延迟为 200 毫秒.这将使您的请求总共完成 819.2 秒而不是 25.6 秒.

This would mean you retry 13 times, with an initial delay of 200ms. This would give your request a total of 819.2s to complete rather than 25.6s.

这篇关于当超过预配置的吞吐量时应该怎么做?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！