Airflow ExternalTaskSensor 执行超时 [英] Airflow ExternalTaskSensor execution timeout
问题描述
我正在使用 airflow.operators.sensors.ExternalTaskSensor
让一个 Dag 等待另一个.
I'm using airflow.operators.sensors.ExternalTaskSensor
to make one Dag wait for another.
dag = DAG(
'dag2',
default_args={
'owner': 'Me',
'depends_on_past': False,
'start_date': start_datetime,
'email': ['me@example.com'],
'email_on_failure': True,
'email_on_retry': False,
'retries': 2,
'retry_delay': timedelta(minutes=10),
},
template_searchpath="%s/me/resources/" % DAGS_FOLDER,
schedule_interval="{} {} * * *".format(minute, hour),
max_active_runs=1
)
wait_for_dag1 = ExternalTaskSensor(
task_id='wait_for_dag1',
external_dag_id='dag1',
external_task_id='dag1_task1',
dag=dag
)
如果上游 Dag 发生严重错误并且未能在给定时间段内完成,我希望上游 Dag(ExternalTaskSensor 运算符)也崩溃,而不是永远挂起.
If something seriously wrong happens with upstream Dag and it fails to complete during the given time period, I want upstream Dag (ExternalTaskSensor operator) crash as well, instead of hanging forever.
如何向 ExternalTaskSensor 添加超时?
我正在查看文档,但它似乎没有 timeout
参数或类似的参数.我该怎么办?
https://airflow.readthedocs.io/en/stable/_modules/airflow/sensors/external_task_sensor.html
I'm looking into documentation, but it does not seem to have a timeout
parameter or something similar. What should I do?
https://airflow.readthedocs.io/en/stable/_modules/airflow/sensors/external_task_sensor.html
推荐答案
ExternalTaskSensor
确实以秒为单位接受 timeout
参数.它从 BaseSensorOperator
(https://airflow.apache.org/docs/apache-airflow/stable/_api/airflow/sensors/base/index.html).如果你在实例化时传递它 timeout=60
,它会在 60 秒后失败.
The ExternalTaskSensor
does take a timeout
argument in seconds. It inherits the argument from BaseSensorOperator
(https://airflow.apache.org/docs/apache-airflow/stable/_api/airflow/sensors/base/index.html). If you pass it timeout=60
on instantiation, it will fail after 60 seconds.
这篇关于Airflow ExternalTaskSensor 执行超时的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!