Django Celery：仅执行一个长时间运行的流程的实例 [英] Django Celery: Execute only one instance of a long-running process

问题描述

我有一个长时间运行的进程，该进程必须每五分钟运行一次，但是永远不应同时运行多个进程实例。该过程通常不应运行超过5分钟，但我想确保第二个实例运行完后不会启动。

I have a long-running process that must run every five minutes, but more than one instance of the processes should never run at the same time. The process should not normally run past five min, but I want to be sure that a second instance does not start up if it runs over.

每先前的建议，我正在使用Django Celery安排这项长期运行的任务。

Per a previous recommendation, I'm using Django Celery to schedule this long-running task.

我认为定期任务不会起作用，因为如果我有五分钟的时间，那么我不希望第二个任务在另一个实例下执行。任务已在运行。

I don't think a periodic task will work, because if I have a five minute period, I don't want a second task to execute if another instance of the task is running already.

我当前的实验如下：在8:55，任务实例开始运行。当任务完成时，它将触发自己的另一个实例在下一个五分钟标记处运行。因此，如果第一个任务在8:57完成，则第二个任务将在9:00运行。如果第一个任务恰好长时间运行并在9:01完成，它将安排下一个实例在9:05运行。

My current experiment is as follows: at 8:55, an instance of the task starts to run. When the task is finishing up, it will trigger another instance of itself to run at the next five min mark. So if the first task finished at 8:57, the second task would run at 9:00. If the first task happens to run long and finish at 9:01, it would schedule the next instance to run at 9:05.

除以下简单示例外，我一直在遇到各种隐秘错误，并且没有发现其他任何人可以通过本身的先前实例。我想知道是否有更好的方法来做我想做的事情。我知道有一种命名任务的方法；也许有一种方法可以搜索具有相同名称的运行中或计划中的实例？对于每五分钟运行一次任务，但确保一次只能运行一个任务，是否有人提供任何建议？

I've been struggling with a variety of cryptic errors when doing anything more than the simple example below and I haven't found any other examples of people scheduling tasks from a previous instance of itself. I'm wondering if there is maybe a better approach to doing what I am trying to do. I know there's a way to name one's tasks; perhaps there's a way to search for running or scheduled instances with the same name? Does anyone have any advice to offer regarding running a task every five min, but ensuring that only one task runs at a time?

谢谢你，
Joe

Thank you, Joe

在mymodule / tasks.py中：

import datetime
from celery.decorators import task 

@task 
def test(run_periodically, frequency):

    run_long_process()
    now = datetime.datetime.now()
    # Run this task every x minutes, where x is an integer specified by frequency
    eta = (
      now - datetime.timedelta(
         minutes =  now.minute % frequency , seconds = now.second, 
         microseconds = now.microsecond ) ) + datetime.timedelta(minutes=frequency) 
    task = test.apply_async(args=[run_periodically, frequency,], eta=eta)  

从./manage.py shell中：

from mymodule import tasks
result = tasks.test.apply_async(args=[True, 5])

推荐答案

您可以使用与特殊锁配对的定期任务ich确保任务一次执行一次。这是Celery文档中的示例实现：

You can use periodic tasks paired with a special lock which ensures the tasks are executed one at a time. Here is a sample implementation from Celery documentation:

> http://ask.github.com/celery/cookbook/tasks.html#ensuring-a-task-is-only-executed-one-一次

您描述的带有从上一次执行开始调度任务的方法，如果其中一个出现故障，则可以停止执行任务。

Your described method with scheduling task from the previous execution can stop the execution of tasks if there will be failure in one of them.

这篇关于Django Celery：仅执行一个长时间运行的流程的实例的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！