如何将新的dag添加到正在运行的气流服务中？ [英] How do I add a new dag to a running airflow service?

问题描述

我有一个气流服务，当前正在作为Web服务器和调度程序的单独docker容器运行，两者均由postgres数据库支持。我让dags在两个实例之间同步，并且在服务启动时dags会正确加载。但是，如果在服务运行时在两个容器上的dag文件夹中添加了一个新的dag，则dag会加载到dagbag中，但会显示在Web gui中且缺少元数据。每次更新后，我都可以运行 airflow initdb，但是感觉不对。调度程序和Web服务器是否有更好的方法与数据库同步？

I have an airflow service that is currently running as separate docker containers for the webserver and scheduler, both backed by a postgres database. I have the dags synced between the two instances and the dags load appropriately when the services start. However, if I add a new dag to the dag folder (on both containers) while the service is running, the dag gets loaded into the dagbag but show up in the web gui with missing metadata. I can run "airflow initdb" after each update but that doesn't feel right. Is there a better way for the scheduler and webserver to sync up with the database?

推荐答案

Dag更新应自动获取。如果他们没有被捡起，通常是因为您所做的更改打破了了。

Dag updates should be picked up automatically. If they don't get picked up, it's often because the change you made "broke" the dag.

要检查新任务是否被捡起，网络服务器，运行：

To check that new tasks are in fact picked up, on your webserver, run:

airflow list_tasks <dag_name> --tree

如果显示未找到Dag，则存在错误。

如果成功运行，则它应显示您的所有任务，并在刷新时在气流ui中拾取这些任务。

If it says Dag not found, then there's an error.
If it runs successfully, then it should show all your tasks and those tasks should be picked up in your airflow ui when you refresh it.

如果有新的/更新的任务没有出现在那儿，然后检查Web服务器上的dags文件夹并验证代码是否确实在更新。

If the new/updated tasks are not showing up there, then check the dags folder on your webserver and verify that the code is indeed being updated.

这篇关于如何将新的dag添加到正在运行的气流服务中？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！