尽管预置容量很高，但 DynamoDB 更新/放置受到限制 [英] DynamoDB Update/Put throttled despite high provisioned capacity

问题描述

我发现我在 DynamoDB 表上的更新受到了一些限制.我知道节流阀每秒工作，有时可以吸收高于配置容量的峰值，但不能保证.我知道应该平均分配负载，我没有这样做.

I am seeing some throttles on my updates on DynamoDB table. I know that throttle work on per second basis, that peaks above provisioned capacity can be sometimes absorbed, but not guaranteed. I know that one is supposed to evenly distribute the load, which I have not done.

但是请查看指标中的 1 分钟平均图表；随附的.使用的容量远低于预置容量.这些油门从哪里来?因为所有的写入都进入了一个特定的分片?

BUT please look at the 1 minute average graphs from metrics; attached. The utilized capacity is way below the provisioned capacity. Where are these throttles coming from? Because all writes went to a particular shard?

没有批量写入.工作负载分布是无法轻易控制的.

There are no batch writes. The workload distribution is something that cannot, easily, control.

推荐答案

DynamoDB 建立在这样的假设之上:为了充分发挥预置吞吐量的潜力，您的读取和写入必须在空间上均匀分布(哈希/范围键)和时间(并非所有都在同一秒出现).

DynamoDB is built on the assumption that to get the full potential out of your provisioned throughput your reads and writes must be uniformly distributed over space (hash/range keys) and time (not all coming in at the exact same second).

根据图表上分配的吞吐量，您仍然很可能位于一个分片上，但如果您之前将吞吐量提高到当前水平以上并将其降低到当前水平，则可能存在两个或更多分片现在.虽然这是需要注意的事情，但它可能不是直接导致这种限制行为的原因.如果您的表中有很多数据，超过 10 GB，那么您肯定会有多个分片.这意味着您的表中可能有很多冷数据，这可能会导致此问题，但这似乎不太可能.

Based on the allocated throughput on your graphs you are still most likely at one shard, but it is possible that there are two or more shards if you have previously raised the throughput above the current level and lowered it down to what it is at now. While this is something to be mindful of, it likely is not what is causing this throttling behavior directly. If you have a lot of data in your table, over 10 GB then you definitely will have multiple shards. This would mean you likely have a lot of cold data in your table and that may be causing this issue, but that seems less likely.

最可能的问题是您有一些热键.具体来说，您有一个或只有几条记录正在接收大量读取或写入请求，这会导致限制.本质上 DynamoDB 可以支持大量的 IOPS 写入和读取，但您不能将所有这些 IOPS 应用到几条记录上，它们需要在理想情况下均匀分布在所有记录中.

The most likely issue is that you have some hot keys. Specifically, you have one or just a few records that are receiving a very high number of read or write requests and this is resulting in throttling. Essentially DynamoDB can support massive IOPS for both writes and reads, but you can't apply all of those IOPS to just a few records, they need to be distributed among all of the records uniformly in an ideal situation.

由于您显示的油门数量在 10 到 100 的数量级之间，因此您不必担心.只要您使用的是官方 AWS 开发工具包，它就会自动处理带有指数退避的重试，以在完全放弃之前多次重试请求.

Since the number of throttles you were showing is in the order of magnitude of 10s to 100s it may not be something to worry about. As long as you are using the official AWS SDK it will automatically take care of retries with exponential backoff to retry requests several times before completely giving up.

虽然在许多情况下很难控制对表的读取和写入分布，但可能值得重新审视一下您的哈希/范围键设计，以确保它真正适合您的读取和写入模式到桌子上.此外，对于读取，您可以通过 Memcached 或 Redis 使用缓存，即使缓存仅在几分钟或几秒钟内过期，以帮助减少热键的影响.对于写入，您需要查看应用程序中的逻辑，以确保没有执行任何可能导致此问题的不必要的写入.

While it is difficult in many circumstances to control the distribution of reads and writes to a table, it may be worth taking another look at your hash/range key design to make sure it is really optimal for your pattern of reads and writes to the table. Also, for reads you may employ caching through Memcached or Redis, even if the cache expired in just a few minutes or a few seconds to help reduce the impact of hot keys. For writes you would need to look at the logic in the application to make sure there are not any unnecessary writes being performed that could be causing this issue.

最后一点与批量写入有关:DynamoDB 中的批量操作不会减少不同子请求所消耗的读取或写入量，它只是减少发出多个 HTTP 请求的开销.虽然批处理请求通常有助于提高吞吐量，但它们在降低 DynamoDB 中出现节流的可能性方面没有用处.

One last point related to batch writes: A batch operation in DynamoDB does not reduce the consumed amount of read or writes the different child requests consume, it simply reduces the overhead of making multiple HTTP requests. While batch requests generally help with throughput, they are not useful at reducing the likelihood of throttling in DynamoDB.

这篇关于尽管预置容量很高，但 DynamoDB 更新/放置受到限制的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！