如何在Airflow中运行bash脚本文件 [英] How to run bash script file in Airflow
本文介绍了如何在Airflow中运行bash脚本文件的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
我有一个bash脚本,用于创建要在Airflow中运行的文件(如果不存在),但是尝试时失败。我该怎么做?
I have a bash script that creates a file (if it does not exist) that I want to run in Airflow, but when I try it fails. How do I do this?
#!/bin/bash
#create_file.sh
file=filename.txt
if [ ! -e "$file" ] ; then
touch "$file"
fi
if [ ! -w "$file" ] ; then
echo cannot write to $file
exit 1
fi
这就是我在气流中的称呼方式:
and here's how I'm calling it in Airflow:
create_command = """
./scripts/create_file.sh
"""
t1 = BashOperator(
task_id= 'create_file',
bash_command=create_command,
dag=dag
)
lib/python2.7/site-packages/airflow/operators/bash_operator.py", line 83, in execute
raise AirflowException("Bash command failed")
airflow.exceptions.AirflowException: Bash command failed
推荐答案
从教程中可以:
t2 = BashOperator(
task_id='sleep',
bash_command='sleep 5',
retries=3,
dag=dag)
但是您正在通过一个对它的多行命令
But you're passing a multi-line command to it
create_command = """
./scripts/create_file.sh
"""
应该
create_command = "./scripts/create_file.sh "
此外,您还必须确保您在正确的目录中,以避免出现神秘错误。例如,这样做:
Moreover, you also have to make sure that you are in the correct directory to avoid cryptic errors. Do it like this for example:
create_command = "./scripts/create_file.sh"
if os.path.exists(create_command):
t1 = BashOperator(
task_id= 'create_file',
bash_command=create_command,
dag=dag
)
else:
raise Exception("Cannot locate {}".format(create_command))
这篇关于如何在Airflow中运行bash脚本文件的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!
查看全文