调试损坏的DAG [英] Debugging Broken DAGs

问题描述

当气流网络服务器显示错误消息，例如损坏的DAG：[< path / to / dag>]< error> 时，我们如何以及在哪里可以找到完整的这些异常的stacktrace吗？

我尝试了以下位置：

/ var / log / airflow / webserver －在执行时间范围内没有日志，其他日志为二进制，并且使用 strings 进行解码没有给出有用的信息。 / p>

/ var / log / airflow / scheduler -有一些日志，但是是二进制形式，试图读取它们

/ var / log / airflow / worker ，并且看起来主要是sqlalchemy日志。显示运行DAG的日志（与您在气流页面上看到的日志相同）

，然后也位于 / var / log / airflow / rotated -找不到我想要的堆栈跟踪。

我正在使用airflow v1.7.1.3

解决方案

通常，我使用命令 airflow list_dags 会打印出在dags中发现的python错误的完整堆栈跟踪。

几乎每当您使用airflow CLI命令时，由于airflow pars dags文件夹都可以使用任何airflow命令。

When the airflow webserver shows up errors like Broken DAG: [<path/to/dag>] <error>, how and where can we find the full stacktrace for these exceptions?

I tried these locations:

/var/log/airflow/webserver -- had no logs in the timeframe of execution, other logs were in binary and decoding with strings gave no useful information.

/var/log/airflow/scheduler -- had some logs but were in binary form, tried to read them and looked to be mostly sqlalchemy logs probably for airflow's database.

/var/log/airflow/worker -- shows up the logs for running DAGs, (same as the ones you see on the airflow page)

and then also under /var/log/airflow/rotated -- couldn't find the stacktrace I was looking for.

I am using airflow v1.7.1.3

解决方案

Usually I used the command airflow list_dags which print the full stacktrace for python error found in dags.

That will work with almost any airflow command as airflow parse dags folder each time you use a airflow CLI command.

这篇关于调试损坏的DAG的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！