如何在Amazon Redshift上执行计划的SQL脚本? [英] How to execute scheduled SQL script on Amazon Redshift?

问题描述

我有一系列大约10个查询，每小时都会在Redshift中自动执行一次(也许会报告成功/失败).

I have series of ~10 queries to be executed every hour automatically in Redshift (maybe report success/failure).

大多数查询都是在我的表上聚合的.

Most queries are aggregation on my tables.

我尝试将AWS Lambda与 CloudWatch Events 一起使用，但是Lambda函数最多只能保留5分钟，而我的查询最多可能需要25分钟.

I have tried using AWS Lambda with CloudWatch Events, but Lambda functions only survive for 5 minutes max and my queries can take up to 25 minutes.

推荐答案

AWS没有提供简单的分布式cron样式服务，这很奇怪.这对于很多事情都是有用的.有 SWF ，但计时/调度方面由用户决定.您可以使用Lambda/Cloudwatch触发SWF事件.要进行合理的cron之类的活动，将花费大量的精力.

It's kind of strange that AWS doesn't provide a simple distributed cron style service. It would be useful for so many things. There is SWF, but the timing/scheduling aspect is left up to the user. You could use Lambda/Cloudwatch to trigger SWF events. That's a lot of overhead to get reasonable cron like activity.

就像评论所说的那样，最简单的方法是运行一个小实例并在其中托管cron作业.使用1的自动缩放组以提高可靠性.一种类似但更复杂的方法是使用弹性beantalk .

Like the comment says the easiest way would be to run a small instance and host cron jobs there. Use an autoscale group of 1 for some reliability. A similar but more complicated approach is to use elastic beanstalk.

如果您真的想要冗余，可靠性，可见性等，那么可能有必要查看第三方解决方案(如 Airflow ).还有许多其他情况取决于您的喜好语言.

If you really want redundancy, reliability, visibility, etc. it might be worth looking at a third party solution like Airflow. There are many others depending on your language of preference.

这是一个类似问题，其中包含更多信息.

Here's a similar question with more info.

这篇关于如何在Amazon Redshift上执行计划的SQL脚本?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！