BashOperator不运行bash文件apache气流 [英] BashOperator doen't run bash file apache airflow
本文介绍了BashOperator不运行bash文件apache气流的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
我刚刚开始使用apache气流。我正在尝试从气流中运行test.sh文件,但是它不起作用。
I just started using apache airflow. I am trying to run test.sh file from airflow, however it is not work.
以下是我的代码,文件名为test.py
Following is my code, file name is test.py
import os
from airflow import DAG
from airflow.operators.bash_operator import BashOperator
from datetime import datetime, timedelta
default_args = {
'owner': 'airflow',
'depends_on_past': False,
'start_date': datetime(2015, 6, 1),
'email': ['airflow@airflow.com'],
'email_on_failure': False,
'email_on_retry': False,
'retries': 1,
'retry_delay': timedelta(minutes=5),
# 'queue': 'bash_queue',
# 'pool': 'backfill',
# 'priority_weight': 10,
# 'end_date': datetime(2016, 1, 1),
}
dag = DAG('test', default_args=default_args)
# t1 and t2 are examples of tasks created by instantiating operators
t1 = BashOperator(
task_id='print_date',
bash_command='date',
dag=dag)
create_command = "sh home/ubuntu/test/inst/scripts/test.sh"
if os.path.exists(create_command):
t2 = BashOperator(
task_id= 'cllTest',
bash_command=create_command,
dag=dag
)
else:
raise Exception("Cannot locate {}".format(create_command))
t2.set_upstream(t1)
当我运行python〜/ airflow / dags / test.py时,它没有抛出任何错误。
when I run python ~/airflow/dags/test.py, it doesn't throw any error.
但是,当我运行气流list_dag时,会引发以下错误:
However, When I run airflow list_dag, it throws following error:
[2017-02-15 20:20:02,741] {__init__.py:36} INFO - Using executor SequentialExecutor
[2017-02-15 20:20:03,070] {models.py:154} INFO - Filling up the DagBag from /home/ubuntu/airflow/dags
[2017-02-15 20:20:03,135] {models.py:2040} ERROR - sh home/ubuntu/test/inst/scripts/test.sh
Traceback (most recent call last):
File "/home/ubuntu/anaconda2/lib/python2.7/site-packages/airflow/models.py", line 2038, in resolve_template_files
setattr(self, attr, env.loader.get_source(env, content)[0])
File "/home/ubuntu/anaconda2/lib/python2.7/site-packages/jinja2/loaders.py", line 187, in get_source
raise TemplateNotFound(template)
TemplateNotFound: sh home/ubuntu/test/inst/scripts/test.sh
我尝试使用如何在Airflow中使用un bash脚本文件寻求答案,
我在哪里弄错了?
推荐答案
在.sh之后添加一个空格,它应该可以运行
,这在气流合流页面
Add a space after .sh it should work this is mentioned in the confluence page of airflow
t2 = BashOperator(
task_id='sleep',
bash_command="/home/batcher/test.sh", // This fails with `Jinja template not found` error
#bash_command="/home/batcher/test.sh ", // This works (has a space after)
dag=dag)
这篇关于BashOperator不运行bash文件apache气流的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!
查看全文