即使主要任务失败也运行和弦回调 [英] Run a chord callback even if the main tasks fail

问题描述

是否可以运行 chord 回调，即使主要任务失败?

我创建了一个和弦，我添加了一堆任务并注册了一个回调给它.我的问题是，如果其中一个任务失败，则不会触发回调，但我希望以任何一种方式触发回调.

我尝试使用 si() 注册回调(不变性)

callback = tasks.run_delete_rule.si([timestamp])header = [tasks.run_update_rule.s(i, timestamp) for i in item_ids]结果 = 和弦(标题)(回调)

我也尝试将参数 ignore_result=True 添加到两个任务装饰器，但没有成功.

解决方案

来自 github issue #1881 if回调设置了 link_error 选项，它接受任务名称列表，然后当和弦的任务失败时，将执行 link_error 任务.

@task(name='super_task.good')定义好():返回真@task(name='super_task.raise_exception')def raise_exception():引发 ValueError('错误')@task(name='super_task.callback')def回调(*args, **kwargs):logger.info('回调')logger.info(args)logger.info(kwargs)返回完成"@task(name='super_task.error_callback')def error_callback(*args, **kwargs):logger.info('error_callback')logger.info(args)logger.info(kwargs)返回错误">>>c = 和弦([raise_exception.s(), good.s(), raise_exception.s()],callback.s().set(link_error=['super_task.error_callback']))>>>结果 = c()

这将执行和弦，在您的 celery 日志中，您将看到 raise_exception 任务失败，并且 error_callback 的执行将在它的 args 中接收 task_id回调.

此时 result 的值将包含 callback 的 AsyncResult 实例，并且因为在和弦中错误会传播到回调执行 result.get() 会引发任务的异常，result.traceback 会给你回溯.

如果你想有一个回调，只需将和弦回调的名称传递给link_error

callback.s().set(link_error='super_task.callback')

注意

另一个选项是设置 CELERY_CHORD_PROPAGATES = False 这将恢复到 celery 3.1 之前的行为并始终执行回调.

但这不是推荐的方法，因为您可以在 github issue #1349

<块引用>

Celery 3.1 定义了如何处理和弦错误，以前的行为从未记录在案更像是一场意外，因为它从来没有打算以这种方式工作.

我们无法更改错误修复版本中的行为，因此必须改用设置，但从来没有人故意禁用新行为.

新行为是为了防止发生此类问题，并且可能会删除向后兼容设置.我建议你在这里找到一些其他的方法来处理错误(如果你能为它发明一个很好的 api，我不介意提出一个建议)

Is it possible to run a chord callback even if the main tasks failed?

I've created a chord which I added a bunch of tasks and registered a callback to it. My problem, is that if one of the tasks fail the callback is not triggered, but I would like the callback to be triggered either way.

I've tried to register the callback with si() (immutability)

callback = tasks.run_delete_rule.si([timestamp])
header = [tasks.run_update_rule.s(i, timestamp) for i in item_ids]
result = chord(header)(callback)

I also tried to add the param ignore_result=True to both tasks decorator, but no success.

解决方案

From the github issue #1881 if the callback has the link_error option set, which takes a list of tasks names, then when a task of the chord fails the link_error tasks are going to be executed.

@task(name='super_task.good')
def good():
    return True

@task(name='super_task.raise_exception')
def raise_exception():
    raise ValueError('error')

@task(name='super_task.callback')
def callback(*args, **kwargs):
    logger.info('callback')
    logger.info(args)
    logger.info(kwargs)
    return 'finished'

@task(name='super_task.error_callback')
def error_callback(*args, **kwargs):
    logger.info('error_callback')
    logger.info(args)
    logger.info(kwargs)
    return 'error'

>>> c = chord(
        [raise_exception.s(), good.s(), raise_exception.s()], 
        callback.s().set(link_error=['super_task.error_callback'])
    )
>>> result = c()

This will execute the chord and in your celery log, you'll see the raise_exception task fail, and the execution of error_callback which will receive in it's args the task_id of callback.

At this point the value of result will contain the AsyncResultinstance of callback, and because in a chord the errors propagate to the callback doing result.get() will raise the exception of the tasks and result.traceback gives you the traceback.

If you want to have a single callback, just pass the name of the chord callback to link_error

callback.s().set(link_error='super_task.callback')

NOTE

Another options it's to set CELERY_CHORD_PROPAGATES = False which will revert to the pre celery 3.1 behavior and always execute the callback.

But this is not a recommended approach because as you can find in the github issue #1349

Celery 3.1 defines how chord errors are handled, the previous behavior was never documented and more of an accident since it was never the intention to work that way.

We couldn't change the behavior in a bugfix release so a setting had to be used instead, but it was never the intention that someone would deliberately disable the new behavior.

The new behavior is there to protect against this sort of issue happening, and the backward compatible setting may be removed. I suggest you find some other way to handle errors here (and I wouldn't mind a proposal if you can invent a nice api for it)

这篇关于即使主要任务失败也运行和弦回调的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！