尽管配置容量高，但DynamoDB更新/放置受到限制 [英] DynamoDB Update/Put throttled despite high provisioned capacity

问题描述

我在DynamoDB表上看到一些更新限制。我知道节流阀是每秒工作的，有时可以吸收但不能保证超过配置容量的峰值。我知道应该平均分配负载，而我还没有完成。

I am seeing some throttles on my updates on DynamoDB table. I know that throttle work on per second basis, that peaks above provisioned capacity can be sometimes absorbed, but not guaranteed. I know that one is supposed to evenly distribute the load, which I have not done.

但是请查看指标的1分钟平均图表；附上。所使用的容量远低于所提供的容量。这些节流阀从哪里来？因为所有写都去了一个特定的分片？

BUT please look at the 1 minute average graphs from metrics; attached. The utilized capacity is way below the provisioned capacity. Where are these throttles coming from? Because all writes went to a particular shard?

没有批量写。工作负载分配是无法轻松控制的。

There are no batch writes. The workload distribution is something that cannot, easily, control.

推荐答案

DynamoDB建立在以下假设之上：要充分利用预配置的吞吐量，您的读写必须均匀地分布在空间（哈希/范围键）上，并且时间（并非所有时间都恰好在同一秒）。

DynamoDB is built on the assumption that to get the full potential out of your provisioned throughput your reads and writes must be uniformly distributed over space (hash/range keys) and time (not all coming in at the exact same second).

根据图上分配的吞吐量，您仍然很可能处于一个碎片中，但是有可能如果以前已将吞吐量提高到当前水平以上并将其降低到现在的水平，则有两个或多个分片。尽管这是要注意的事情，但可能不是直接导致这种节流行为的原因。如果表中有大量数据，超过10 GB，则肯定会有多个分片。这将意味着您的表中可能有很多冷数据，这可能会导致此问题，但可能性似乎较小。

Based on the allocated throughput on your graphs you are still most likely at one shard, but it is possible that there are two or more shards if you have previously raised the throughput above the current level and lowered it down to what it is at now. While this is something to be mindful of, it likely is not what is causing this throttling behavior directly. If you have a lot of data in your table, over 10 GB then you definitely will have multiple shards. This would mean you likely have a lot of cold data in your table and that may be causing this issue, but that seems less likely.

最可能的问题是您有一些热键。具体地说，您有一个或几个记录正在接收大量的读取或写入请求，这会导致节流。本质上，DynamoDB可以支持大量的IOPS来进行读写操作，但是您不能将所有IOPS都应用于少数记录，在理想情况下，它们需要在所有记录之间均匀分布。

The most likely issue is that you have some hot keys. Specifically, you have one or just a few records that are receiving a very high number of read or write requests and this is resulting in throttling. Essentially DynamoDB can support massive IOPS for both writes and reads, but you can't apply all of those IOPS to just a few records, they need to be distributed among all of the records uniformly in an ideal situation.

由于您显示的油门数量大约在10s到100s的数量级之间，因此不必担心。只要您使用的是官方的AWS开发工具包，它就会自动处理具有指数回退的重试，以在完全放弃之前重试几次请求。

Since the number of throttles you were showing is in the order of magnitude of 10s to 100s it may not be something to worry about. As long as you are using the official AWS SDK it will automatically take care of retries with exponential backoff to retry requests several times before completely giving up.

在许多情况下，要控制对表的读写分布，可能值得再次看一下哈希/范围键设计，以确保它对于表的读写模式确实是最佳的。此外，对于读取，您可以通过Memcached或Redis进行缓存，即使缓存在几分钟或几秒钟内过期也可以帮助减少热键的影响。对于写操作，您需要查看应用程序中的逻辑，以确保没有执行任何可能导致此问题的不必要写操作。

While it is difficult in many circumstances to control the distribution of reads and writes to a table, it may be worth taking another look at your hash/range key design to make sure it is really optimal for your pattern of reads and writes to the table. Also, for reads you may employ caching through Memcached or Redis, even if the cache expired in just a few minutes or a few seconds to help reduce the impact of hot keys. For writes you would need to look at the logic in the application to make sure there are not any unnecessary writes being performed that could be causing this issue.

最后一点是相关的到批处理写入：DynamoDB中的批处理操作不会减少读取的消耗量或写入不同的子请求所消耗的消耗量，它只是减少了发出多个HTTP请求的开销。虽然批处理请求通常有助于提高吞吐量，但它们对于减少DynamoDB中限制的可能性没有用。

One last point related to batch writes: A batch operation in DynamoDB does not reduce the consumed amount of read or writes the different child requests consume, it simply reduces the overhead of making multiple HTTP requests. While batch requests generally help with throughput, they are not useful at reducing the likelihood of throttling in DynamoDB.

这篇关于尽管配置容量高，但DynamoDB更新/放置受到限制的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！