将数据从Amazon Aurora同步到Redshift [英] Sync data from Amazon Aurora to Redshift

问题描述

我正在尝试在AWS Aurora和Redshift之间建立同步.实现同步的最佳方法是什么?

可能的同步方式可以是:-

1. 查询表以查找表中的更改(因为我仅执行插入操作，更新无关紧要)，将这些更改导出到S3存储桶中的平面文件，并使用Redshift copy命令插入到Redshift中./p>

2. 使用 python Publisher 和Boto3将更改发布到Kinesis流中然后在Firehose中使用此流，从那里可以直接复制到Redshift中.

3. 使用Kinesis Agent检测binlog中的更改(可以使用Kinesis Agent检测int binlog中的更改)并将其发布到Firehose，然后从那里复制到Firehose中.

我还没有探索过AWS Datapipeline.

解决方案

@Mark B指出， Amazon Aurora可以成为源和 Amazon Redshift可以成为目标

I am trying to setup a sync between AWS Aurora and Redshift. What is the best way to achieve this sync?

Possible ways to sync can be: -

1. Query table to find changes in a table(since I am only doing inserts, updates don't matter), export these changes to a flat file in S3 bucket and use Redshift copy command to insert into Redshift.

2. Use python publisher and Boto3 to publish changes into a Kinesis stream and then consume this stream in Firehose from where I can copy directly into Redshift.

3. Use Kinesis Agent to detect changes in binlog (Is it possible to detect changes int binlog using Kinesis Agent) and publish it to Firehose and from there copy into Firehose.

I haven't explored AWS Datapipeline yet.

解决方案

As pointed out by @Mark B, the AWS Database Migration Service can migrate data between databases. This can be done as a one-off exercise, or it can run continuously, keeping two databases in sync.

The documentation shows that Amazon Aurora can be a source and Amazon Redshift can be a target.

这篇关于将数据从Amazon Aurora同步到Redshift的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！