重试芹菜链失败的任务 [英] Retrying celery failed tasks that are part of a chain

问题描述

我有一条芹菜链可以执行一些任务。每个任务都可能失败并被重试。请参见下面的快速示例：

I have a celery chain that runs some tasks. Each of the tasks can fail and be retried. Please see below for a quick example:

from celery import task

@task(ignore_result=True)
def add(x, y, fail=True):
    try:
        if fail:
            raise Exception('Ugly exception.')
        print '%d + %d = %d' % (x, y, x+y)
    except Exception as e:
        raise add.retry(args=(x, y, False), exc=e, countdown=10)

@task(ignore_result=True)
def mul(x, y):
    print '%d * %d = %d' % (x, y, x*y)

和链条：

from celery.canvas import chain
chain(add.si(1, 2), mul.si(3, 4)).apply_async()

运行两个任务（并假设没有任何失败） ），您将获得/看到打印内容：

Running the two tasks (and assuming that nothing fails), your would get/see printed:

1 + 2 = 3
3 * 4 = 12

但是，当添加任务第一次失败并在s中成功时频繁重试调用，链中的其余任务均未运行，即添加任务失败，链中的所有其他任务均未运行，几秒钟后，添加任务再次运行并成功，其余任务

However, when the add task fails the first time and succeeds in subsequent retry calls, the rest of the tasks in the chain do not run, i.e. the add task fails, all other tasks in the chain are not run and after a few seconds, the add task runs again and succeeds and the rest of the tasks in the chain (in this case mul.si(3, 4)) does not run.

芹菜提供了一种从失败的任务继续失败的链的方法，向前？如果不是，那么最好的方法是做到这一点，并确保链中的任务以指定的顺序运行，并且仅在上一个任务成功执行后才能执行，即使该任务重试了几次也是如此？

Does celery provide a way to continue failed chains from the task that failed, onwards? If not, what would be the best approach to accomplishing this and making sure that a chain's tasks run in the order specified and only after the previous task has executed successfully even if the task is retried a few times?

注1：可以通过以下操作解决该问题

Note 1: The issue can be solved by doing

add.delay(1, 2).get()
mul.delay(3, 4).get()

但我有兴趣了解为什么链不能用于失败的任务。

but I am interested in understanding why chains do not work with failed tasks.

推荐答案

您发现了一个错误：）

已修复 https://github.com/celery/celery/commit/b2b9d922fdaed5571cf685249bdc46f28acacde3
将是3.0.4的一部分。

Fixed in https://github.com/celery/celery/commit/b2b9d922fdaed5571cf685249bdc46f28acacde3 will be part of 3.0.4.

这篇关于重试芹菜链失败的任务的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！