气流任务失败/重试工作流程 [英] Airflow Task failure/retry workflow

问题描述

我对任务有重试逻辑，还不清楚打开重试后Airflow如何处理任务失败。

I have retry logic for tasks and it's not clear how Airflow handles task failures when retries are turned on.

他们的文档仅声明 on_failure_callback 在任务失败时被触发，但是如果该任务失败并且也标记为重试，是否意味着将同时调用 on_failure_callback 和 on_retry_callback ？

Their documentation just states that on_failure_callback gets triggered when a task fails, but if that task fails and is also marked for retry does that mean that both the on_failure_callback and on_retry_callback would be called?

推荐答案

重试逻辑/参数将在失败逻辑/参数之前发生。因此，如果您将任务设置为重试两次，它将在失败（然后执行）之前再次尝试运行两次（并执行 on_retry_callback ） on_failure_callback ）。

Retry logic/parameters will take place before failure logic/parameters. So if you have a task set to retry twice, it will attempt to run again two times (and thus executing on_retry_callback ) before failing (and then executing on_failure_callback).

一种简单的确认执行顺序的方法是设置 email_on_retry 和 email_on_failure 到 True 并查看它们出现的顺序。您可以从物理上确认它会在失败之前重试。

An easy way to confirm the sequence that it is executed in is to set your email_on_retry and email_on_failure to True and see the order in which they appear. You can physically confirm that it will retry before failing.

default_args = {
    'owner': 'me',
    'start_date': datetime(2019, 2, 8),
    'email': ['[e-mail]'],
    'email_on_failure': True,
    'email_on_retry': True,
    'retries': 1,
    'retry_delay': timedelta(minutes=1)
}

这篇关于气流任务失败/重试工作流程的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！