AWS:在redshift中自动化查询 [英] AWS: Automating queries in redshift

问题描述

我想自动执行每天要运行的redshift插入查询.

I want to automate a redshift insert query to be run every day.

我们实际上使用Aws环境.有人告诉我使用lambda是不正确的方法.这是在Redshift中自动执行查询的最佳ETL过程.

We actually use Aws environment. I was told using lambda is not the right approach. Which is the best ETL process to automate a query in Redshift.

推荐答案

要在Redshift上自动执行SQL，您至少要有3个选择

For automating SQL on Redshift you have 3 options (at least)

简单-cron 使用EC2实例并在该实例上设置cron作业以运行您的SQL代码.

Simple - cron Use a EC2 instance and set up a cron job on that to run your SQL code.

psql -U youruser -p 5439 -h hostname_of_redshift -f your_sql_file

功能丰富-气流(推荐) 如果您要执行复杂的计划，那么值得花时间学习和使用apache气流.这也需要在服务器(ec2)上运行，但是提供了很多功能.

Feature rich - Airflow (Recommended) If you have a complex schedule to run then it is worth investing time learning and using apache airflow. This also needs to run on a server(ec2) but offers a lot of functionality.

https://airflow.apache.org/

无AWS的AWS-AWS数据管道(不推荐)

https://aws.amazon.com/datapipeline/

Cloudwatch-> Lambda-> EC2方法，下面由John Rotenstein描述 当您想以AWS为中心时，这是一个好方法，它将比拥有专用的EC2实例便宜.

Cloudwatch->Lambda->EC2 method described below by John Rotenstein This is a good method when you want to be AWS centric, it will be cheaper than having a dedicated EC2 instance.

这篇关于AWS:在redshift中自动化查询的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！