如何保持多个独立的芹菜队列？ [英] How to keep multiple independent celery queues?

问题描述

我正在尝试将多个具有不同任务和工作人员的芹菜队列保留在同一Redis数据库中。真的只是一个方便问题，我的机器上只需要一个Redis服务器，而不是两个。

I'm trying to keep multiple celery queues with different tasks and workers in the same redis database. Really just a convenience issue of only wanting one redis server rather than two on my machine.

我逐字遵循celery教程文档，因为这是获得它的唯一方法为我工作。现在，当我尝试使用稍有调整的名称/队列来复制所有内容时，它总是会出错。

I followed the celery tutorial docs verbatim, as it as the only way to get it to work for me. Now when I try to duplicate everything with slightly tweaked names/queues, it keeps erroring out.

注意-我是Python和Celery的新手，这显然是一部分问题。我不确定哪个部分的名称与特殊词相对应。

Note - I'm a newish to Python and Celery, which is obviously part of the problem. I'm not sure which parts are named "task/tasks" as a name vs special words.

我的文档精简版：
Run celery-任务工作人员生成工作人员。
tasks.py包含任务代码，其中 celery = Celery（'tasks'，broker ='redis：// localhost'）连接到Celery和 @task（）我要延迟的功能之上。

My condensed version of docs: Run celery -A tasks worker to spawn the workers. tasks.py contains task code with celery = Celery('tasks', broker='redis://localhost') to connect to Celery and @task() above my functions that I want to delay.

在我的程序中，用于排队任务...

Within my program for queueing tasks...

from tasks import do_work
do_work.delay()

因此，鉴于上述所有情况，我需要采取哪些步骤将其转变为两种类型的任务，分别在单独的队列和工作线程上运行？例如，blue_tasks和red_tasks？

So given all of the above, what are the steps I need to take to turn this into two types of tasks that run independently on separate queues and workers? For example, blue_tasks and red_tasks?

我尝试将所有任务实例更改为blue_tasks或red_tasks。但是，当我将blue_tasks排队时，我开始使用的red_tasks工作程序开始尝试对其进行处理。

I've tried changing all instances of tasks to blue_tasks or red_tasks. However, when I queue blue_tasks, the red_tasks workers I've started up start trying to work on them.

我了解了默认队列等信息，因此我尝试了此代码，但无效：

I read about default queues and such, so I tried this code, which didn't work:

CELERY_DEFAULT_QUEUE = 'red'
CELERY_QUEUES = (
    Queue('red', Exchange('red'), routing_key='red'),
)

作为旁注，我不明白为什么 celery worker 会因芹菜尝试连接到默认amqp实例而出错，而 celery -A任务工作者告诉celery连接到Redis。如果未指定任何内容，芹菜工人试图在该工人上运行什么任务代码？

As a side note, I don't understand why celery worker errors out with celery attempting to connect to a default amqp instance, while celery -A tasks worker tells celery to connect to Redis. What task code is celery worker attempting to run on the worker if nothing has been specified?

推荐答案

默认情况下，所有内容都进入名为 celery 的默认队列中（这就是 celery worker 将要执行的操作）

By default everything goes into a default queue named celery (and this is what celery worker will process if no queue is specified)

因此，假设您在中具有 do_work 任务功能django_project_root / myapp / tasks.py 。

So say you have your do_work task function in django_project_root/myapp/tasks.py.

您可以将 do_work 任务配置为

CELERY_ROUTES = {
    'myproject.tasks.do_work': {'queue': 'red'},
}

然后使用 celery worker -Q red ，它将仅处理该队列中的内容（另一个由 celery worker 调用的工作者将仅在默认情况下拾取内容队列）

Then run a worker using celery worker -Q red and it will only process things in that queue (another worker invoked with celery worker will only pickup things in the default queue)

任务路由秒文档中的所有说明都应该说明。

The task routing section in the documentation should explain all.

这篇关于如何保持多个独立的芹菜队列？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！