如何在 Python 脚本中触发气流 DAG 运行? [英] How to trigger an airflow DAG run from within a Python script?

问题描述

使用 apache 气流，我创建了一些 DAGS，其中一些不按计划运行.
我正在尝试找到一种方法，可以从 Python 脚本中触发特定 DAG 的运行.这可能吗?我能怎么做?

Using apache airflow, I created some DAGS, some of which do not run on a schedule.
I'm trying to find a way that I can trigger a run for a specific DAG from within a Python script. Is this possible? How can I do?

EDIT --- python 脚本将从与我所有 DAGS 所在的项目不同的项目中运行

EDIT --- The python script will be running from a different project from the project where all my DAGS are located

推荐答案

在触发 Airflow DAG 运行时，您有多种选择.

You have a variety of options when it comes to triggering Airflow DAG runs.

airflow python 包提供了一个本地客户端 您可以用于从 python 脚本中触发 dag.例如:

The airflow python package provides a local client you can use for triggering a dag from within a python script. For example:

from airflow.api.client.local_client import Client

c = Client(None, None)
c.trigger_dag(dag_id='test_dag_id', run_id='test_run_id', conf={})

使用 Airflow CLI

您可以使用 Airflow CLI 手动触发气流中的 dag.有关如何使用 CLI 触发 DAG 的更多信息，请参见此处.

您还可以使用 Airflow REST api 来触发 DAG 运行.有关更多信息这里.

You can also use the Airflow REST api to trigger DAG runs. More info on that here.

python 中的第一个选项可能最适合您(这也是我过去亲自完成的方式).但理论上您可以使用 subprocess 与来自 python 的 CLI 交互，或者requests 之类的库，用于在 Python 中与 REST API 交互.

The first option from within python might work for you best (it's also how I've personally done it in the past). But you could theoretically use a subprocess to interact with the CLI from python, or a library like requests to interact with the REST API from within Python.

这篇关于如何在 Python 脚本中触发气流 DAG 运行?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！