每两周安排一次气流工作 [英] Schedule airflow job bi-weekly

问题描述

我有一个要求，我想每隔一个星期五安排一次气流作业.但是，问题是我不知道如何为此编写时间表.

I have a requirement that I want to schedule an airflow job every alternate Friday. However, the problem is I am not able to figure out how to write a schedule for this.

我不想为此有多个工作.

I don't want to have multiple jobs for this.

我试过了

'0 0 1-7,15-21 * 5

但是它不起作用，它每天从 1 点到 7 点和 15 点到 21 点运行.

However it's not working it's running from 1 to 7 and 15 to 21 everyday.

从 shubham 的回答中，我意识到我们可以拥有一个 PythonOperator，它可以为我们跳过任务.我试图实施解决方案.但是似乎不起作用.

from shubham's answer I realize that we can have a PythonOperator which can skip the task for us. An I tried to implement the solution. However doesn't seem to work.

因为在 2 周内进行测试太难了.这就是我所做的.

As testing this on 2 week period would be too difficult. This is what I did.

• 我安排 DAG 每 5 分钟运行一次
• 但是，我正在编写 python 操作符跳过替代任务(与我正在尝试做的事情非常相似，替代星期五).

DAG:

args = {
    'owner': 'Gaurang Shah',
    'retries': 0,
    'start_date':airflow.utils.dates.days_ago(1),
}


dag = DAG(
    dag_id='test_dag',
    default_args=args,
    catchup=False,
    schedule_interval='*/5 * * * *',
    max_active_runs=1
    )


dummy_op = DummyOperator(task_id='dummy', dag=dag)

def _check_date(execution_date, **context):
    min_date = datetime.now() - relativedelta(minutes=10)
    print(context)
    print(context.get("prev_execution_date"))
    print(execution_date)
    print(datetime.now())
    print(min_date)
    if execution_date > min_date:
        raise AirflowSkipException(f"No data available on this execution_date ({execution_date}).")

check_date = PythonOperator(
    task_id="check_if_min_date",
    python_callable=_check_date,
    provide_context=True,
    dag=dag,
)

推荐答案

根据您的实现，您可以使用散列.使用 1.10 版在我的气流计划中工作:

Depending on your implementation you can use the hash. Worked in my airflow schedules using version 1.10:

哈希(#)'#' 允许用于星期字段，并且后面必须跟一个 1 到 5 之间的数字.它允许指定诸如第二个星期五"之类的结构.给定月份的.[19]例如，输入5#3"在星期几字段对应于每个月的第三个星期五.参考

Hash (#) '#' is allowed for the day-of-week field, and must be followed by a number between one and five. It allows specifying constructs such as "the second Friday" of a given month.[19] For example, entering "5#3" in the day-of-week field corresponds to the third Friday of every month. Reference

这篇关于每两周安排一次气流工作的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！