DynamoDB 中原子计数器的可靠性 [英] Reliability of atomic counters in DynamoDB

问题描述

我正在考虑在我的应用程序中使用 Amazon DynamoDB，但我对它的 原子计数器可靠性.

I was considering to use Amazon DynamoDB in my application, and I have a question regarding its atomic counters reliability.

我正在构建一个分布式应用程序，它需要同时和一致递增/递减存储在 Dynamo 属性中的计数器.我想知道 Dynamo 的原子计数器在重并发环境中的可靠性如何，其中并发级别非常高(假设，例如，平均 20k 并发命中率 - 明白这一点，这将是近 520 亿的增量/每月递减).

I'm building a distributed application that needs to concurrently, and consistently, increment/decrement a counter stored in a Dynamo's attribute. I was wondering how reliable the Dynamo's atomic counter is in an heavy concurrent environment, where the concurrency level is extremely high (let's say, for example, an average rate of 20k concurrent hits - to get the idea, that would be almost 52 billions increments/decrements per month).

计数器应该是超级可靠的，并且永远不会错过任何命中.有人在如此关键的环境中测试过 DynamoDB 吗?

The counter should be super-reliable and never miss a hit. Has somebody tested DynamoDB in such critical environments?

谢谢

推荐答案

DynamoDB 通过在多个服务器之间拆分密钥来获得它的扩展属性.这类似于 Cassandra 和 HBase 等其他分布式数据库的扩展方式.虽然您可以增加 DynamoDB 上的吞吐量，只需将您的数据移动到多个服务器，现在每个服务器都可以处理总并发连接数/服务器数量.查看他们的常见问题解答 有关如何实现最大吞吐量的说明:

DynamoDB gets it's scaling properties by splitting the keys across multiple servers. This is similar to how other distributed databases like Cassandra and HBase scale. While you can increase the throughput on DynamoDB that just moves your data to multiple servers and now each server can handle total concurrent connections / number of servers. Take a look at their FAQ for an explanation on how to achieve max throughput:

问:我是否总是能够达到我的预置吞吐量水平?

Amazon DynamoDB 假定所有主键的访问模式相对随机.您应该设置您的数据模型，以便您的请求导致跨主键的流量分布相当均匀.如果您的访问模式非常不均匀或有偏差，您可能无法达到您的预置吞吐量水平.

Amazon DynamoDB assumes a relatively random access pattern across all primary keys. You should set up your data model so that your requests result in a fairly even distribution of traffic across primary keys. If you have a highly uneven or skewed access pattern, you may not be able to achieve your level of provisioned throughput.

在存储数据时，Amazon DynamoDB 将一个表划分为多个分区，并根据主键的哈希键元素分布数据.与表关联的预置吞吐量也在分区之间分配；每个分区的吞吐量根据分配给它的配额独立管理.没有跨分区共享预配置吞吐量.因此，如果工作负载相当均匀地分布在哈希键值上，Amazon DynamoDB 中的表最能满足预置的吞吐量水平.跨哈希键值分布请求会跨分区分布请求，这有助于实现完全预置的吞吐量水平.

When storing data, Amazon DynamoDB divides a table into multiple partitions and distributes the data based on the hash key element of the primary key. The provisioned throughput associated with a table is also divided among the partitions; each partition's throughput is managed independently based on the quota allotted to it. There is no sharing of provisioned throughput across partitions. Consequently, a table in Amazon DynamoDB is best able to meet the provisioned throughput levels if the workload is spread fairly uniformly across the hash key values. Distributing requests across hash key values distributes the requests across partitions, which helps achieve your full provisioned throughput level.

如果您的主键工作负载模式不均衡，并且无法达到您预置的吞吐量级别，您可以通过进一步提高预置吞吐量级别来满足您的吞吐量需求，这将为每个分区提供更多吞吐量.但是，建议您考虑修改请求模式或数据模型，以实现跨主键的相对随机访问模式.

If you have an uneven workload pattern across primary keys and are unable to achieve your provisioned throughput level, you may be able to meet your throughput needs by increasing your provisioned throughput level further, which will give more throughput to each partition. However, it is recommended that you considering modifying your request pattern or your data model in order to achieve a relatively random access pattern across primary keys.

这意味着拥有一个直接递增的密钥将无法扩展，因为该密钥必须位于一台服务器上.还有其他方法可以解决此问题，例如在内存聚合中使用 DynamoDB 的刷新增量(尽管这可能存在可靠性问题)或分片计数器，其中增量分布在多个键上并通过拉取分片中的所有键来读取计数器(http://whynosql.com/scaling-distributed-counters/).

This means that having one key that is incremented directly will not scale since that key must live on one server. There are other ways to handle this problem, for example in memory aggregation with a flush increment to DynamoDB (though this can have reliability issues) or a sharded counter where the increments are spread over multiple keys and read back by pulling all keys in the sharded counter (http://whynosql.com/scaling-distributed-counters/).

这篇关于DynamoDB 中原子计数器的可靠性的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！