如何为DynamoDB将超过25个项目/行写入Table？ [英] How to write more than 25 items/rows into Table for DynamoDB?

问题描述

我对Amazon DynamoDB相当陌生。我目前有20000行需要添加到表中。但是，根据我所阅读的内容，使用BatchWriteItem类和25个WriteRequests似乎一次最多只能写25行。有可能增加吗？如何一次写超过25行？目前大约需要15分钟才能写入所有20000行。谢谢。

I am quite new to Amazon DynamoDB. I currently have 20000 rows that I need to add to a table. However, based on what I've read, it seems that I can only write up to 25 rows at a time using BatchWriteItem class with 25 WriteRequests. Is it possible to increase this? How can I write more than 25 rows at a time? It is currently taking about 15 minutes to write all 20000 rows. Thank you.

推荐答案

在一个BatchWriteItem请求中最多只能发送25个项目，但是您可以发送尽可能多的BatchWriteItem请求一次即可。假设您已经提供了足够的写入吞吐量，您应该能够通过在多个线程/进程/主机之间拆分这2万行并将它们并行地推送到数据库来显着加快处理速度。

You can only send up to 25 items in a single BatchWriteItem request, but you can send as many BatchWriteItem requests as you want at one time. Assuming you've provisioned enough write throughput, you should be able to speed things up significantly by splitting those 20k rows between multiple threads/processes/hosts and pushing them to the database in parallel.

数据集，但您可以使用 AWS数据管道从S3提取数据。它基本上可以自动化创建Hadoop集群的过程，以从S3提取数据并将其通过一堆并行的BatchWriteItem请求发送到DynamoDB。

It's maybe a bit heavyweight for that small of a dataset, but you can use AWS Data Pipeline to ingest data from S3. It basically automates the process of creating a Hadoop cluster to suck down your data from S3 and send it to DynamoDB in a bunch of parallel BatchWriteItem requests.

这篇关于如何为DynamoDB将超过25个项目/行写入Table？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！