仅将新记录从AWS DynamoDB复制到AWS Redshift [英] Copying only new records from AWS DynamoDB to AWS Redshift

问题描述

我看到有很多示例和文档将数据从DynamoDB复制到Redshift，但是我们正在研究一个增量复制过程，其中只有新行从DynamoDB复制到Redshift。我们将每天运行此复制过程，因此无需每天杀死整个redshift表。有人对此主题有任何经验或想法吗？

I see there is tons of examples and documentation to copy data from DynamoDB to Redshift, but we are looking at an incremental copy process where only the new rows are copied from DynamoDB to Redshift. We will run this copy process everyday, so there is no need to kill the entire redshift table each day. Does anybody have any experience or thoughts on this topic?

推荐答案

Dynamo DB具有称为流：

Dynamo DB has a feature (currently in preview) called Streams:

Amazon DynamoDB Streams在日志中的任何DynamoDB表中维护项目
级更改的时间顺序，持续时间为24个
小时。使用Streams API，开发人员可以查询更新，
在更改前后接收项目级别的数据，并使用
为其在
DynamoDB之上构建的应用程序构建创意扩展。

Amazon DynamoDB Streams maintains a time ordered sequence of item level changes in any DynamoDB table in a log for a duration of 24 hours. Using the Streams APIs, developers can query the updates, receive the item level data before and after the changes, and use it to build creative extensions to their applications built on top of DynamoDB.

此功能将使您能够处理新的更新，并根据需要进行操作，而不是设计导出系统在DynamoDB之上。

This feature will allow you to process new updates as they come in and do what you want with them, rather than design an exporting system on top of DynamoDB.

您可以在读取和处理DynamoDB流文档。

这篇关于仅将新记录从AWS DynamoDB复制到AWS Redshift的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！