重试气流期间如何保持值 [英] How to persist a value during airflow retry

问题描述

每次dag触发任务时，我都想生成一个值并在重试期间保留该值(如果任务由于任何原因而无法运行)

Everytime a dag triggers a task I would like to generate a value and keep the value during retry (if the task fails to run for any reason)

我尝试在dag中声明一个变量(基本上将当前时间设置为该变量) 当任务失败并重试时，它会再次将变量设置为最新时间

I try to declare a variable inside the dag (basically set current time to the variable) When the task fails and it retries, it set the variable with latest time again

推荐答案

• 您可以使用BaseOperator的> on_failure_callback和on_retry_callback参数将要保留的值作为 xcom

  • You can make use of on_failure_callback and on_retry_callback params of BaseOperator to push the value that you want to persist as xcom

    此后，您可以在您的Airflow部署的任何DAG中的任何task中，从Xcom 检索数据，包括相同或不同的task

    Thereafter, you can retrieve that data from Xcom in any task across any DAG of your Airflow deployment, including the retry attempt of same / different task

    也看看

    • 参数更改后的Airflow DAG任务能否重试

    Airflow Task失败/重试工作流程

    这篇关于重试气流期间如何保持值的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！