清除上游任务后的气流重新运行，即使成功则下游也可以 [英] Airflow force re-run of upstream task when cleared even though downstream if marked success

问题描述

我在Airflow中有任务A-> B-> C，当我运行DAG并成功完成所有任务时，我希望能够单独清除B（同时将C标记为成功）。 B清除并进入 no_status状态，但是当我尝试重新运行B时，什么也没发生。我尝试了--ignore_dependencies，-ignore_depends_on_past和--force，但无济于事。如果C也已清除，那么B似乎只会重新运行，然后一切都按预期方式重新运行。

I have tasks A -> B -> C in Airflow and when I run the DAG and all complete with success, I'd like to be able to clear B alone (while leaving C marked as success). B clears and gets put into the 'no_status' state but then when I try to re-run B, nothing happens. I've tried --ignore_dependencies, --ignore_depends_on_past and --force but to no avail. B seems to only re-run if C is also cleared and then everything re-runs as expected.

我希望能够重新运行B的原因特别是在不更改管道的情况下，B的某些外部输入可能会稍有变化（文件已更改或调整），我想在重新启动下游任务之前运行它并评估其输出（以减轻任何潜在的中断）。

The reason why I'd like to be able re-run B specifically without changing the pipeline is that some of B's external inputs may change slightly (file changed, or tweak) and I'd like to run it and evaluate it's output before restarting the downstream tasks (to mitigate any potential interruption).

推荐答案

如果通过命令行清除，则可以指定要清除的任务。

If you clear via the command line, you can specify which tasks you want cleared.

airflow clear your_dag_id --task_regex B --start_date 2017-09-25T17:00:00 --end_date 2017-09-25T17:00:00

如果要通过UI进行此操作，则可以转到浏览->。任务实例查找要清除的任务实例。从那里删除仅应清除您选择的任务。您还需要转到浏览->。 DAG Run ，并将相关的DAG运行设置为运行状态，以供您执行已清除的任务。

If you want to do this via the UI, you can go to Browse -> Task Instances to find the task instance you want to clear. Deleting from there should only clear the tasks you selected. You will also need to go over to Browse -> DAG Run and set the relevant DAG run to the running state for your cleared task to be picked up.

这篇关于清除上游任务后的气流重新运行，即使成功则下游也可以的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！