使用相同的args/kwarg撤消芹菜任务 [英] Revoke celery tasks with same args/kwargs

问题描述

想象一下，如果要执行长时间的任务，并且要有一组特定的args和kwargs.在启动新任务之前，是否有机会以相同的args/kwargs撤销所有正在运行和待处理的任务，因为Im只对最后添加的任务的结果感兴趣.(底层数据在两次调用之间发生变化)

Imagine having a long running task with a specific set of args and kwargs. Is there any chance to revoke all running and pending tasks with the same args/kwargs before starting a new task as Im only interested in the result of the last added task. (The underlying data changes inbetween two calls)

我尝试迭代 inspect.active()， inspect.registered()和 inspect.scheduled()的结果以获取全部任务，然后 filter/revoke 那些带有我的args和kwargs的任务.

I tried iterating the results of inspect.active(), inspect.registered() and inspect.scheduled() to get ALL tasks and then filter/revoke those with my args and kwargs in question.

但这并不可靠，因为检查所有工人和寻找任务的时间太长了.

But this was not reliable as the inspecting of all workers and searching for tasks took way too long.

任何人都可以使我朝正确的方向前进吗?

Anyone can get me in the right direction?

推荐答案

如果检查工作人员速度较慢，则最好通过辅助数据存储(例如像redis这样的键值存储)进行检查.您为功能添加了一个锁"，因此您知道它已经启动了.

If inspecting the workers is to slow, it might be better to check via a secondary data storage, for example a key-value store like redis. You add an "lock" for an function, so you know you already started it.

排队任务时:

• 检查商店中是否已经存在
  • 是:引发异常
  • 否:将相关任务信息作为项目添加到您的商店.

  一个工人将执行它

  • 执行它
  • 从存储中删除

  有一个基于Redis的现成实现:

  There is a redis based ready made implementation:

  芹菜一次 (您必须将其指定为任务的 base

  You have to specify it as a base of a task

  from celery import Celery
  from celery_once import QueueOnce
  
  @celery.task(base=QueueOnce)
  def sum(a, b):
      ...
  

  您还可以指定要考虑的参数和其他一些设置，有关详细信息，请阅读自述文件.

  You can also specify which parameters to take into account and some more settings, read the readme there for more details.

  这篇关于使用相同的args/kwarg撤消芹菜任务的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！