如何设置Airflow的电子邮件配置以发送有关错误的电子邮件？ [英] How do I setup Airflow's email configuration to send an email on errors?

问题描述

我试图通过传递无效的Bash行（此不应该运行）来使Airflow任务故意失败和出错。气流输出以下内容：

I'm trying to make an Airflow task intentionally fail and error out by passing in a Bash line (thisshouldnotrun) that doesn't work. Airflow is outputting the following:

[2017-06-15 17:44:17,869] {bash_operator.py:94} INFO - /tmp/airflowtmpLFTMX7/run_bashm2MEsS: line 7: thisshouldnotrun: command not found
[2017-06-15 17:44:17,869] {bash_operator.py:97} INFO - Command exited with return code 127
[2017-06-15 17:44:17,869] {models.py:1417} ERROR - Bash command failed
Traceback (most recent call last):
  File "/home/ubuntu/.local/lib/python2.7/site-packages/airflow/models.py", line 1374, in run
    result = task_copy.execute(context=context)
  File "/home/ubuntu/.local/lib/python2.7/site-packages/airflow/operators/bash_operator.py", line 100, in execute
    raise AirflowException("Bash command failed")
AirflowException: Bash command failed
[2017-06-15 17:44:17,871] {models.py:1433} INFO - Marking task as UP_FOR_RETRY
[2017-06-15 17:44:17,878] {models.py:1462} ERROR - Bash command failed
Traceback (most recent call last):
  File "/home/ubuntu/.local/bin/airflow", line 28, in <module>
    args.func(args)
  File "/home/ubuntu/.local/lib/python2.7/site-packages/airflow/bin/cli.py", line 585, in test
    ti.run(ignore_task_deps=True, ignore_ti_state=True, test_mode=True)
  File "/home/ubuntu/.local/lib/python2.7/site-packages/airflow/utils/db.py", line 53, in wrapper
    result = func(*args, **kwargs)
  File "/home/ubuntu/.local/lib/python2.7/site-packages/airflow/models.py", line 1374, in run
    result = task_copy.execute(context=context)
  File "/home/ubuntu/.local/lib/python2.7/site-packages/airflow/operators/bash_operator.py", line 100, in execute
    raise AirflowException("Bash command failed")
airflow.exceptions.AirflowException: Bash command failed

Airflow是否会针对此类错误发送电子邮件？如果没有，发送这些错误电子邮件的最佳方法是什么？

Will Airflow send an email for these kind of errors? If not, what would be the best way to send an email for these errors?

我什至不确定 airflow.cfg 已正确设置...由于最终目标是测试电子邮件警报通知，因此我想确保airflow.cfg已正确设置。设置如下：

I'm not even sure if airflow.cfg is setup properly... Since the ultimate goal is to test the email alerting notification, I want to make sure airflow.cfg is setup properly. Here's the setup:

[email]
email_backend = airflow.utils.email.send_email_smtp


[smtp]
# If you want airflow to send emails on retries, failure, and you want to use
# the airflow.utils.email.send_email_smtp function, you have to configure an
# smtp server here
smtp_host = emailsmtpserver.region.amazonaws.com 
smtp_starttls = True
smtp_ssl = False
# Uncomment and set the user/pass settings if you want to use SMTP AUTH
# smtp_user = airflow_data_user
# smtp_password = password
smtp_port = 587 
smtp_mail_from = [e-mail]

什么是 smtp_starttls ？我在文档或在线中找不到任何信息。如果我们需要两要素身份验证才能查看电子邮件，那么这是否会成为Airflow的问题？

What is smtp_starttls? I can't find any info for it in the documentation or online. If we have 2-factor authentication needed to view emails, will that be an issue here for Airflow?

这是我的Bash命令：

Here's my Bash command:

task1_bash_command = """
export PATH=/home/ubuntu/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin
export rundate=`TZ='America/Los_Angeles' date +%F -d "yesterday"`
export AWS_CONFIG_FILE="/home/ubuntu/.aws/config"

/home/ubuntu/bin/snowsql -f //home/ubuntu/sql/script.sql 1> /home/ubuntu/logs/"$rundate"_dev.log 2> /home/ubuntu/logs/"$rundate"_error_dev.log

if [ -e /home/ubuntu/logs/"$rundate"_error_dev.log ]
then
    exit 64
fi

我的任务：

task1 = BashOperator(
    task_id = 'run_bash',
    bash_command = task1_bash_command,
    dag = dag,
    retries = 2,
    email_on_failure = True,
    email = '[e-mail]')

推荐答案

smtp_starttls 基本上意味着使用 TLS

将此设置为 False 并设置 smtp_ssl 到 True ，如果您想使用SSL。您可能都需要 smtp_user 和 smtp_password 。

Set this to False and set smtp_ssl to True if you want to use SSL instead. You probably need smtp_user and smtp_password for either.

气流将不处理两步身份验证。但是，如果您使用的是AWS，则您可能不需要它，因为您的SMTP（SES）凭证与您的AWS凭证不同。

Airflow will not handle 2 step authentication. However, is you are using AWS you likely don't need it as your SMTP (SES) credentials are different from your AWS credentials.

请参阅此处。

编辑：
要使气流在失败时发送电子邮件，您需要在任务中设置以下几项： email_on_failure 和电子邮件。

例如：

def throw_error(**context):
    raise ValueError('Intentionally throwing an error to send an email.')



t1 = PythonOperator(task_id='throw_error_and_email',
                    python_callable=throw_error,
                    provide_context=True,
                    email_on_failure=True,
                    email='[e-mail]',
                    dag=dag)

这篇关于如何设置Airflow的电子邮件配置以发送有关错误的电子邮件？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！