从S3存储桶流式传输数据以定期进行红移 [英] Stream data from S3 bucket to redshift periodically

问题描述

我在S3中存储了一些数据.我需要定期将数据从S3克隆/复制到Redshift集群.要进行批量复制，我可以使用copy命令从S3复制到redshift.

I have some data stored in S3 . I need to clone/copy this data periodically from S3 to Redshift cluster. To do bulk copy , I can use copy command to copy from S3 to redshift.

类似地，有任何简单的方法可以将数据从S3定期复制到Redshift.

Similarly is there any trivial way to copy data from S3 to Redshift periodically .

谢谢

推荐答案

AWS Lambda Redshift Loader 是一个很好的解决方案，只要新文件出现在Amazon上的预配置位置，它就会在Redshift上运行COPY命令S3.

AWS Lambda Redshift Loader is a good solution that runs a COPY command on Redshift whenever a new file appears pre-configured location on Amazon S3.

链接:

https://aws.amazon.com/blogs/big-data/a-zero-administration-amazon-redshift-database-loader/ https://github.com/awslabs/aws-lambda-redshift-loader

这篇关于从S3存储桶流式传输数据以定期进行红移的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！