我不能通过 BashOperator xcom_push 参数 [英] I can't xcom_push an arguments through BashOperator

问题描述

我是 Airflow 的 xcom 功能的新手.我用 PythonOperator 进行了尝试，它工作正常(即，我可以将值从上下文中推入和拉出)，但是当我在 BashOperator 上尝试时，它不起作用.但是，我只能通过在任务创建期间添加 xcom_push=True 属性来提取最终的 stdout 语句.这是一回事.2)但我也希望像我们在 PythonOp 中那样根据它们的键(到 BashOp 和从 BashOp)推送和拉取值.这真的很有帮助，因为我需要将大量变量从一个脚本传递到另一个.

I am new to the Airflow's xcom feature. i tried it out with PythonOperator and it was working fine(i.e., i can push and pull the value out of the context), but when i tried it out on BashOperator, it didn't work. However i can pull only the final stdout statement by adding the xcom_push=True attribute during the task creation. that's one thing. 2) But i also wish to push and pull the values based on their keys (to and from the BashOp) like the way we do it in PythonOp.. It would be really helpful since i need to pass tons of variables from one script to another.

推荐答案

这是您想要的吗?

from datetime import datetime
from airflow.models import DAG
from airflow.operators.bash_operator import BashOperator

dag = DAG(
    dag_id="example_bash_operator_1",
    schedule_interval=None,
    start_date=datetime(2018, 12, 31),
)

t1 = BashOperator(
    task_id="t1",
    bash_command='echo "{{ ti.xcom_push(key="k1", value="v1") }}" "{{ti.xcom_push(key="k2", value="v2") }}"',
    dag=dag,
)

t2 = BashOperator(
    task_id="t2",
    bash_command='echo "{{ ti.xcom_pull(key="k1") }}" "{{ ti.xcom_pull(key="k2") }}"',
    dag=dag,
)

t1 >> t2

这篇关于我不能通过 BashOperator xcom_push 参数的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！