Dag似乎失踪了 [英] Dag Seems to be missing
问题描述
我有一个dag，它定期检查要生成的新工作流程（动态DAG），如果找到，则创建它们。 （请参阅：调度程序未添加动态dags）
I have a dag which checks for new workflows to be generated (Dynamic DAG) at a regular interval and if found, creates them. (Ref: Dynamic dags not getting added by scheduler )
以上DAG正在运行，动态DAG正在创建并在Web服务器中列出。这里有两个问题：
The above DAG is working and the dynamic DAGs are getting created and listed in the web-server. Two issues here:
- 在Web URL中单击DAG时，它说 DAG似乎丢失了。
- 未使用 airflow list_dags命令列出列出的DAG
错误：
DAG app01_user似乎丢失。
DAG "app01_user" seems to be missing.
所有其他动态生成的DAG。我已经编译了Python脚本，没有发现错误。
The same is for all other dynamically generated DAGs. I have compiled the Python script and found no errors.
Edit1：
我试图清除所有数据并运行 airflow run。它成功运行，但没有将动态生成的DAG添加到 airflow list_dags中。但是，当运行命令 airflow list_dags时，它会加载并执行DAG（生成动态DAG）。动态DAG也列出如下：
I tried clearing all data and running "airflow run". It ran successfully but no Dynamic generated DAGs were added to "airflow list_dags". But when running the command "airflow list_dags", it loaded and executed the DAG, (which generated Dynamic DAGs). The dynamic DAGs are also listed as below:
[root@cmnode dags]# airflow list_dags
sh: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8\nLANG=en_US.UTF-8)
sh: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8\nLANG=en_US.UTF-8)
[2019-08-13 00:34:31,692] {settings.py:182} INFO - settings.configure_orm(): Using pool settings. pool_size=15, pool_recycle=1800, pid=25386
[2019-08-13 00:34:31,877] {__init__.py:51} INFO - Using executor LocalExecutor
[2019-08-13 00:34:32,113] {__init__.py:305} INFO - Filling up the DagBag from /root/airflow/dags
/usr/lib/python2.7/site-packages/airflow/operators/bash_operator.py:70: PendingDeprecationWarning: Invalid arguments were passed to BashOperator (task_id: tst_dyn_dag). Support for passing such arguments will be dropped in Airflow 2.0. Invalid arguments were:
*args: ()
**kwargs: {'provide_context': True}
super(BashOperator, self).__init__(*args, **kwargs)
-------------------------------------------------------------------
DAGS
-------------------------------------------------------------------
app01_user
app02_user
app03_user
app04_user
testDynDags
再次运行后，以上所有生成的4 dag消失了，只有基本的DAG，显示 testDynDags。
Upon running again, all the above generated 4 dags disappeared and only the base DAG, "testDynDags" is displayed.
推荐答案
当我收到此错误时，Web服务器日志中显示了一个异常。一旦我解决了该错误并重新启动了网络服务器，它就可以正常通过。
When I was getting this error, there was an exception showing up in the webserver logs. Once I resolved that error and I restarted the webserver it went through normally.
从我的视线中可以看到，这是网络服务器尝试解析dag时引发的错误。文件，并且有一个错误。就我而言，导入我添加到插件的新运算符是一个错误。
From what I can see this is the error that is thrown when the webserver tried to parse the dag file and there is an error. In my case it was an error importing a new operator I added to a plugin.
这篇关于Dag似乎失踪了的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
