气流:具有一项任务的简单 DAG 永远不会完成 [英] Airflow: Simple DAG with one task never finishes

问题描述

我制作了一个非常简单的 DAG，如下所示:

I have made a very simple DAG that looks like this:

from datetime import datetime
from airflow import DAG
from airflow.operators.bash_operator import BashOperator

cleanup_command = "/home/ubuntu/airflow/dags/scripts/log_cleanup/log_cleanup.sh "

dag = DAG(
'log_cleanup',
description='DAG for deleting old logs',
schedule_interval='10 13 * * *',
start_date=datetime(2018, 3, 30),
catchup=False,
)

t1 = BashOperator(task_id='cleanup_task', bash_command=cleanup_command, dag=dag)

任务成功完成，但尽管如此，DAG 仍处于运行"状态.知道什么可能导致这种情况.下面的屏幕截图显示了 DAG 保持运行的问题.较早的运行仅完成，因为我手动将状态标记为成功.

The task finishes successfully but despite of this the DAG remains in "running" status. Any idea what could cause this. The screenshot below show the issue with the DAG remaining running. The earlier runs are only finished because I manually mark status as success.

推荐答案

之前的运行仅完成，因为我手动将状态设置为运行.

The earlier runs are only finished because I manually set status to running.

您确定您的调度程序正在运行吗?您可以使用 $airflow scheduler 启动它，并检查 scheduler CLI 命令docs 您不必手动将任务设置为正在运行.

Are you sure your scheduler is running? You can start it with $ airflow scheduler, and check the scheduler CLI command docs You shouldn't have to manually set tasks to running.

你这里的代码看起来不错.您可以尝试的一件事是重新启动您的调度程序.

Your code here seems fine. One thing you might try is restarting your scheduler.

在 Airflow 元数据数据库中，DAG 运行结束状态与任务运行结束状态断开连接.我以前见过这种情况，但通常当它意识到 DAG 运行中的所有任务都已达到最终状态(成功、失败或跳过)时，它会在调度程序的下一个循环中自行解决.

In the Airflow metadata database, DAG run end state is disconnected from task run end state. I've seen this happen before, but usually it resolves itself on the scheduler's next loop when it realizes all of the tasks in the DAG run have reached a final state (success, failed, or skipped).

您是否在此处运行 LocalExecutor、SequentialExecutor 或其他东西?

Are you running the LocalExecutor, SequentialExecutor, or something else here?

这篇关于气流:具有一项任务的简单 DAG 永远不会完成的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！