Celery:确保工作人员不只运行一个用户的作业 [英] Celery: Make sure workers are not running only jobs from one user

问题描述

我有4位芹菜工人，并发率为6.

I have 4 celery workers each with concurrency of 6.

我有用户提交不同数量的作业(从1到20).

I have users submitting varying number of jobs (from 1 to 20).

如何确保每个用户的作业获得相同的处理时间，并且确保一个用户的作业不会填满队列，从而迫使其他用户的作业等待.

How do I ensure that each user's job get equal processing time, and that one user's job do not fill up the queue forcing other user's jobs to wait.

恐怕如果工人最终要完成第一个用户提交的所有工作，那么其他用户排队的工作必须等待第一个用户完成，这给您带来不便.

I am afraid if the workers are ending up going through all the jobs submitted by the first user, the other user's queued jobs must wait first user to finish, an inconvenience.

有没有一种方法可以使芹菜工作者意识到一个用户的工作阻止了另一个用户的工作.相反，我可以在任何给定时间从每个用户运行最多一项作业吗?

Is there a way to make the celery workers aware of one user's jobs holding up other user's queued jobs . Instead can I run maximum one job from each user at any given time?

我有一个将所有用户作业提交到的队列，是否需要为每个用户创建一个队列，并且采用某种循环策略从每个用户的队列中提取一个作业?

I have one queue which I submit all the user's jobs to, would I need to make a queue for each user and somehow have round-robin strategy to pull one job from each user's queue?

推荐答案

此刻为每个用户创建队列并基于循环算法对其进行调度似乎是一项繁重的工作.

Making a queue for each user and scheduling them based on round-robin algorithm seems to be a lot of work.

解决问题的一种简单方法是创建一个临时表&存储传入的任务详细信息.将第一个收到的任务发送到芹菜.到完成时，您可能已经从各种用户那里收到了许多任务.现在，根据用户ID，完成任务&未完成的任务，您可以将大多数适当的任务发送给celery进行执行.

One simple way to solve your problem is to create a temporary table & store the incoming task details. Send first received task to celery. By the time it's completed, you might have received a lot of tasks from various users. Now based on the user id, completed tasks & uncompleted tasks, you can send most appropriate task to celery for execution.

这篇关于Celery:确保工作人员不只运行一个用户的作业的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！