如何将超过 25 个项目/行写入 DynamoDB 表? [英] How to write more than 25 items/rows into Table for DynamoDB?

问题描述

我对 Amazon DynamoDB 很陌生.我目前有 20000 行需要添加到表中.但是，根据我所读到的内容，我似乎一次只能使用带有 25 个 WriteRequests 的 BatchWriteItem 类最多写入 25 行.有可能增加这个吗?如何一次写入超过 25 行?目前写入所有 20000 行大约需要 15 分钟.谢谢.

I am quite new to Amazon DynamoDB. I currently have 20000 rows that I need to add to a table. However, based on what I've read, it seems that I can only write up to 25 rows at a time using BatchWriteItem class with 25 WriteRequests. Is it possible to increase this? How can I write more than 25 rows at a time? It is currently taking about 15 minutes to write all 20000 rows. Thank you.

推荐答案

在一个 BatchWriteItem 请求中最多只能发送 25 个项目，但一次可以发送任意多个 BatchWriteItem 请求.假设您已经提供了足够的写入吞吐量，您应该能够通过在多个线程/进程/主机之间拆分这 20k 行并将它们并行推送到数据库来显着加快速度.

You can only send up to 25 items in a single BatchWriteItem request, but you can send as many BatchWriteItem requests as you want at one time. Assuming you've provisioned enough write throughput, you should be able to speed things up significantly by splitting those 20k rows between multiple threads/processes/hosts and pushing them to the database in parallel.

对于这么小的数据集，它可能有点重量级，但您可以使用 AWS Data Pipeline 从 S3 摄取数据.它基本上自动化了创建 Hadoop 集群的过程，以从 S3 中吸取您的数据，并通过一组并行 BatchWriteItem 请求将其发送到 DynamoDB.

It's maybe a bit heavyweight for that small of a dataset, but you can use AWS Data Pipeline to ingest data from S3. It basically automates the process of creating a Hadoop cluster to suck down your data from S3 and send it to DynamoDB in a bunch of parallel BatchWriteItem requests.

这篇关于如何将超过 25 个项目/行写入 DynamoDB 表?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！