阻止Celery Beat运行同一任务 [英] Prevent Celery Beat from running the same task

问题描述

我每30秒安排一次芹菜运行任务。我有一个每天作为任务运行，另一个每周在用户指定的时间和星期几运行。它检查开始时间和下一个预定日期。下一个计划的日期直到任务完成后才会更新。

I have a scheduled celery running tasks every 30 seconds. I have one that runs as task daily, and another one that runs weekly on a user specified time and day of the week. It checks for the "start time" and the "next scheduled date". The next scheduled date does not update until the task is completed.

但是，我想知道如何确保芹菜节拍只运行一次任务。我现在看到，芹菜将多次运行某个任务，直到该任务的下一个计划日期已更新。

However, I want to know how to make sure that the celery beat is only running the task once. I see that right now, celery will run a certain task multiple times until that task's next scheduled date has been updated.

推荐答案

在为了做到这一点，您需要实现某种分布式锁定，而解决此问题的一种简单可靠的方法是使用带有memcached后端的django缓存，并在任务开始时才在任务完成之前在其中设置标志删除该标志。另一种选择是将 redis锁用作分布式锁。
使用django缓存memcached作为后端的示例：

In order to do that you need to implement some kind of "distributed lock", and easy an reliable approach to this issue is to use the django cache with memcached backend and set a "flag" in it when the task starts then just before it finish remove that flag. Other option is to use "redis" lock as "distributed lock". Example of using django cache memcached as backend:

@shared_task
def my_task(arg1, arg2, lock_expire=300):
    lock_key = 'my_task'
    acquire_lock = lambda: cache.add(lock_key, '1', lock_expire)
    release_lock = lambda: cache.delete(lock_key)

    if acquire_lock():
        try:
            # Execute your code here!
        except Exception:
            # error handling here
        finally:
            # release allow other task to execute
            release_lock()
    else:
        print("Other task is running, skipping")

上面的代码实现了分布式锁以确保不管您尝试执行多少次，都只能运行一个任务。
锁只能由一项任务获得：)，​​另一项将跳过主块并完成。
对您来说有意义吗？

The code above implements a "distributed lock" to ensure that only one task could run, no matter how many times you try to execute it again. The lock can be acquired by only one task :), the other will just skip the "main block" and finish. Does it make sense to you?

玩得开心！

这篇关于阻止Celery Beat运行同一任务的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！