为什么 Airflow 没有在下一个预定时间启动新的 Dag? [英] Why Airflow didn't start the new Dag on the next scheduled time?

问题描述

例如，我在 11:30 创建了一个新的 DAG.DAG 应该在每小时的第一分钟运行(Cron:0 * * * *).我预计 DAG 会在 12:00 开始.然而，DAG 并于 13:00 首次启动.

这里发生了什么?

查看后端Mysqldag表，新dag的next_dagrun列显示12:00.

解决方案

引自 Airflow 文档:

<块引用>

如果您以一天的 schedule_interval 运行 DAG，则执行日期为 2019-11-21 的运行将在 2019-11-21T23:59 之后不久触发.

<块引用>

让我们重复一遍，调度程序在开始日期之后的一个 schedule_interval 期间运行您的作业.

想想就好像实际开始日期是指前一个执行日期，间隔过去后会发生 dag 运行.因此，当您在 12:00 看到 next_dagrun 时，它将在 13:00 执行(DAG Runs UI 上的 Start Date 列)，等等

换句话说，如果您在 11:30 创建了 DAG，那么您的第一个 schedule_interval 在 12:00 还没有完成.对我来说，最终了解发生了什么的诀窍是使用 start_date 参数.

希望对你有用！

For example, I created a new DAG on 11:30. The DAG supposed to run on the first minute of every hour (Cron: 0 * * * *). I expected the DAG start on 12:00. However the DAG and first started on 13:00.

What is going on here?

I checked the backend Mysql dag table, the next_dagrun column of the new dags shows 12:00.

解决方案

Quote from the Airflow docs:

If you run a DAG on a schedule_interval of one day, the run with execution_date 2019-11-21 triggers soon after 2019-11-21T23:59.

Let’s Repeat That, the scheduler runs your job one schedule_interval AFTER the start date, at the END of the period.

Think about it as if it's the actual start date refers to the previous execution date, dag runs will happen after the interval is passed. So when you see next_dagrun at 12:00, that will get executed (Start Date column on DAG Runs UI) at 13:00 and so on.

In other words, if you created the DAG at 11:30, at 12:00 your first schedule_interval wasn't completed yet. For me the trick to finally understand what was going on, was to play around with the start_date param.

Hope that works for you!

这篇关于为什么 Airflow 没有在下一个预定时间启动新的 Dag?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！