如何从rabbitMQ永久删除芹菜任务? [英] How do I permanently remove a celery task from rabbitMQ?

问题描述

我当前的芹菜设置中约有10,000个计划任务.我不知道预定的任务是什么，于是决定使用它们提前几个月发送后续电子邮件.

I have around 10,000 scheduled tasks on my current celery setup. I didn't realize what scheduled tasks were and decided to use them to send follow-up emails months in advance.

回想一下，将来安排一个任务多于一个小时可能永远不是一个好主意，因为每次重新启动工作程序时，它都必须重新接收来自RabbitMQ的每个计划任务，然后它们都只能坐在记忆.

Looking back, it's probably never a good idea to schedule a task for more than 1 hour in the future as every time you restart a worker it has to re-receive every scheduled task from rabbitMQ and then they all just sit in the memory.

我的问题是，如果我必须撤消一项任务，它不仅会删除它.任务保留在内存中，但是撤销队列现在包含任务的ID.准备执行时，芹菜会检查它是否已被撤销，如果已被撤销，它会在此时将其撤销.

My problem is that if I have to revoke a task, it doesn't just delete it. The task stays in memory but a revoke queue now contains the ID of the task. When it is up for execution, celery checks to see if it is revoked and if it is, it will revoke it at this point.

但是，任务将一直保留在内存中，直到那时，如果我随时重启我的工作器，由于我没有将其永久保留，因此撤消队列将被清除.

However, the task will still stay in memory up until then, and if I restart my worker at anytime, the revoke queue will be cleared as I didn't make it persistent.

如何从芹菜工作者中永久删除任务?我基本上只需要将确认的邮件发送回rabbitMQ，以便rabbit一劳永逸地将其删除，如果我重新启动celery，它将不会回来.

我已经查看了文档和源代码，并尝试在shell中自己完成此操作，但是我无法确定将任务挂接到rabbitMQ并永久弹出的正确位置.

I've looked in the docs and source code and tried to do it myself in the shell but I can't figure out the proper place for acking a task to rabbitMQ and then popping it forever.

推荐答案

如果您仅使用一个队列或一项任务，这很容易:

从答案:您可以使用celery purge命令清除所有已配置的任务队列:

Answer: You can use the celery purge command to purge all configured task queues:

$ celery -A proj purge

或以编程方式:

>>> from proj.celery import app
>>> app.control.purge()
1753

如果您只想从特定队列中清除消息，则必须使用AMQP API或celery amqp实用程序:

If you only want to purge messages from a specific queue you have to use the AMQP API or the celery amqp utility:

$ celery -A proj amqp queue.purge <queue name>

数字1753是已删除邮件的数量.

The number 1753 is the number of messages deleted.

您还可以使用--purge参数启动工作程序，以在工作程序启动时清除消息.

You can also start worker with the --purge argument, to purge messages when the worker starts.

更新: 如果您有多个队列或任务

我不知道在RabbitMQ中对其进行编辑的任何方式，因为服务器并不是以这种方式访问​​/编辑/删除排队的任务的，但是您始终可以在代码中禁用任务:

I don't know of any way to edit them in RabbitMQ since the server is not built to access/edit/delete queued tasks that way, but you could always disable your task in the code:

@task
def my_old_task()
   pass

这样，所有任务将按计划运行，但不执行任何操作；由于它们既未重命名也未删除，因此不会有任何错误.

This way all the tasks will run as scheduled but won't perform anything; since they are neither renamed nor deleted, you won't have any error.

显然，您应该更新代码以停止计划这些任务.片刻之后，将不再计划此类任务，因此您可以删除代码.

Obviously you should update your code to stop scheduling these tasks. After a while, there won't be anymore tasks of this type scheduled so you can delete the code.

这篇关于如何从rabbitMQ永久删除芹菜任务?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！