DAG在Airflow中完成运行后如何删除XCOM对象 [英] How to delete XCOM objects once the DAG finishes its run in Airflow

问题描述

我在XCOM中有一个巨大的json文件，稍后dag执行完成后就不需要了，但是我仍然在UI中看到包含所有数据的Xcom对象，有没有办法以编程方式删除XCOM DAG运行完成。

I have a huge json file in the XCOM which later I do not need once the dag execution is finished, but I still see the Xcom Object in the UI with all the data, Is there any way to delete the XCOM programmatically once the DAG run is finished.

谢谢

推荐答案

您必须添加任务取决于您的元数据数据库（sqllite，PostgreSql，MySql ..）可在DAG运行完成后删除XCOM。

You have to add a task depends on you metadatadb (sqllite, PostgreSql, MySql..) that delete XCOM once the DAG run is finished.

delete_xcom_task = PostgresOperator(
      task_id='delete-xcom-task',
      postgres_conn_id='airflow_db',
      sql="delete from xcom where dag_id=dag.dag_id and 
           task_id='your_task_id' and execution_date={{ ds }}",
      dag=dag)

您可以验证您的

数据分析->临时查询-> airflow_db->查询->运行！

Data Profiling -> Ad Hoc Query -> airflow_db -> query -> Run!

这篇关于DAG在Airflow中完成运行后如何删除XCOM对象的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！