如何在 EmailOperator 任务的文件名中添加模板变量?(空气流动) [英] How to add template variable in the filename of an EmailOperator task? (Airflow)

问题描述

我似乎无法让它发挥作用.

I can't seem to get this to work.

我正在尝试每天发送一个给定的文件，其名称类似于file_{{ds_nodash}}.csv".

I am trying to send daily a given file, whose name is like 'file_{{ds_nodash}}.csv'.

问题是我似乎无法将此名称添加为文件名，因为它似乎无法使用.在电子邮件的文本或主题中完美无缺，而不是在名称上.

The problem is that I can't seem to add this name as the filename, since it seems it cant be used. In the text of the email or the subject works perfectly, not not on the name.

以 dag 为例:

local_file = 'file-{{ds_nodash}}.csv'

send_stats_csv = EmailOperator(
    task_id='send-stats-csv',
    to=['[e-mail]'],
    subject='Subject - {{ ds }}',
    html_content='Here is the new file.',
    files=[local_file],
    dag=dag)

错误代码:没有这样的文件或目录:u'file-{{ds_nodash}}.csv'

Error code: No such file or directory: u'file-{{ds_nodash}}.csv'

如果我按字面意思写，加上给定的日期，它就完美无缺.

If i write it literally, with its given date, it works flawlessly.

我哪里错了?我应该怎么做?

Where am I wrong? How should I go about this?

任何帮助将不胜感激.

谢谢.

PD从气流的文档中复制粘贴 - 默认情况下，气流引擎会传递一些可在所有模板中访问的变量".https://airflow.incubator.apache.org/code.html

P.D. Copy paste from airflow's documentation - "The Airflow engine passes a few variables by default that are accessible in all templates". https://airflow.incubator.apache.org/code.html

如果我理解正确，这些变量在执行中是可以访问的，所以如果我正在执行 dag，应该可以找到该文件对吗?我试过测试任务或回填 dag 都没有成功.

If I understood correctly, these variables are accessible in execution, so if i am executing the dag, the file should be found right? I've tried both testing the task or backfilling the dag with no success.

推荐答案

Airflow Operators 定义哪些字段是模板字段.对于 EmailOperator，只有主题和 html_content 字段被设置为模板.

Airflow Operators define what fields are template fields. For the EmailOperator only the subject and html_content fields are set as templates.

class EmailOperator(BaseOperator):
    ...
    template_fields = ('subject', 'html_content')
    template_ext = ('.html',)

参见:https://airflow.incubator.apache.org/_modules/email_operator.html

来自 Airflow Gotcha 的页面 (https://gtoonstra.github.io/etl-with-airflow/gotchas.html)

From the Airflow Gotcha's Page (https://gtoonstra.github.io/etl-with-airflow/gotchas.html)

不是所有运算符中的参数都是模板化的，所以你不能在任何地方使用 Jinja 模板.Jinja 模板仅适用于在 template_fields... 中列出的运算符中的那些字段.

Not all parameters in operators are templated, so you cannot use Jinja templates everywhere. The Jinja templates only work for those fields in operators where it’s listed in the template_fields...

要使其正常工作，您必须从 EmailOperator 派生一个新类，并为文件数组添加模板.

To get this to work, you would have to derive a new class from EmailOperator and add in templating for the files array.

这篇关于如何在 EmailOperator 任务的文件名中添加模板变量?(空气流动)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！