增加DynamoDB中的分片数量以并行启动更多lambda [英] Increase number of shards in DynamoDB to spin up more lambdas in parallel

问题描述

我当前正在使用 DynamoDB 流来通过lambda函数处理更改的集合值，但是，目前，我仅并行运行两个lambda实例，这不足以处理所有传入的数据和lambda函数刚刚排队。

I'm currently using DynamoDB streams to process changed collection values with lambda functions, however, currently, I'm only running two lambda instances in parallel, which is not enough to process all the incoming data and lambda functions are just queued up.

从 aws 文档中，我可以看到可以并行运行的lambda数量与DynamoDB的分片数量成正比：

From aws documentation I can see that number of lambdas that can run in parallel is proportional to the number of shards of your DynamoDB:

如果创建一个Lambda函数来处理基于
的基于流的服务（Amazon Kinesis Streams或DynamoDB流）中的事件，则
每个流中的分片数是并发单位。如果您的
流具有100个活动分片，则将同时运行100个Lambda函数
。然后，每个Lambda函数都按照事件到达的顺序在
碎片上处理事件。

If you create a Lambda function that processes events from stream-based services (Amazon Kinesis Streams or DynamoDB streams), the number of shards per stream is the unit of concurrency. If your stream has 100 active shards, there will be 100 Lambda functions running concurrently. Then, each Lambda function processes events on a shard in the order that they arrive.

所以我的问题是，如何如何增加DynamoDB的分片数量？可能吗我在设置中找不到如何设置它。

So my question is, how do I increase the number of shards of my DynamoDB? Is it even possible? I couldn't find how to set it up in the settings.

推荐答案

否，无法手动控制DDB UpdateStream中的分片数量。
DDB通过创建尽可能多的分片以匹配更新的传入速率来自动为您处理该问题。

No, its not possible to manually control number of shards in DDB UpdateStream. DDB automatically handles that for you by creating as many shards to match the incoming rate of updates.

理想情况下，DDB表中发生的更新应该能够通过一些分片（对同一记录进行的更新将始终使用相同的分片，这意味着它们将基于您的hashKey进行分区）。
也是您的更新流，也按时间顺序排列，因此同一记录上的更新最终以相同的分片结束（或说排队），以便最终处理器按顺序处理它们发生的情况。

Ideally updates happening to your DDB table is supposed to flow through some shard (updates happening to same record will always go to same shard meaning they are partitioned based on your hashKey). It is your stream of updates that too in chronological order thus updates over same record end up (or say queued up) in same shard so that end processor process them in sequence they happened.

每个分片都有其自己的数据输入和输出吞吐量能力，除非需要更多分片来支持即将到来的表更新速率（在DDB更新流的情况下，您的tps写入率很高）表格，当前无法处理的分片数量）

Each shard has its own throughput capacity for in and out of data unless there is need of more shards to support in coming rate of updates on table (which in case of DDB updates streams is high write tps on your table, which current number of shards can't handle)

这篇关于增加DynamoDB中的分片数量以并行启动更多lambda的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！