重试芹菜任务并以指数方式退回 [英] Retry Celery tasks with exponential back off

问题描述

对于这样的任务

from celery.decorators import task

@task()
def add(x, y):
    if not x or not y:
        raise Exception("test error")
    return self.wait_until_server_responds(

如果它引发异常，并且我想从守护程序端重试，如何应用指数退避算法，即在 2 ^ 2、2 ^ 3,2 ^ 4 等秒之后？

if it throws an exception and I want to retry it from the daemon side, how can apply an exponential back off algorithm, i.e. after 2^2, 2^3,2^4 etc seconds?

服务器端，这样如果该工作人员正好被杀死，那么下一个产生的工作人员将执行重试任务？

Also is the retry maintained from the server side, such that if the worker happens to get killed then next worker that spawns will take the retry task?

推荐答案

a href = http://celery.readthedocs.org/en/latest/userguide/tasks.html#context> task.request.retries 属性包含到目前为止的尝试次数，
，因此您可以使用它来实现指数补偿：

The task.request.retries attribute contains the number of tries so far, so you can use this to implement exponential back-off:

from celery.task import task

@task(bind=True, max_retries=3)
def update_status(self, auth, status):
    try:
        Twitter(auth).update_status(status)
    except Twitter.WhaleFail as exc:
        self.retry(exc=exc, countdown=2 ** self.request.retries)

防止雷声群问题，您可以考虑在指数补偿中添加随机抖动：

To prevent a Thundering Herd Problem, you may consider adding a random jitter to your exponential backoff:

import random
self.retry(exc=exc, countdown=int(random.uniform(2, 4) ** self.request.retries))

这篇关于重试芹菜任务并以指数方式退回的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！