使用 Supervisor 运行多个 Laravel 队列工作者 [英] Running multiple Laravel queue workers using Supervisor

问题描述

我使用 Laravel 队列，使用数据库驱动程序和主管来保持队列工作器始终运行:

I using Laravel queues using a database driver and supervisor to keep a queue worker running all the time:

[program:laravel_queue]
command=php artisan queue:listen --timeout=1800 --tries=5
directory=/var/app/current
stdout_logfile=/var/app/support/logs/laravel-queue.log
logfile_maxbytes=0
logfile_backups=0
redirect_stderr=true
autostart=true
autorestart=true
startretries=86400
EOB

某些队列任务可能需要大约 10 分钟才能完成.

Some of the queue tasks can take around 10 minutes to complete.

我有两个问题:

1) 我如何编辑上述脚本以在同一个队列上运行多个(例如 3 个)队列工作人员.

1) How can i edit the above script to run multiple (e.g. 3) queue workers on the same queue.

2) 有没有办法根据等待处理的作业数量来扩展正在运行的队列工作人员的数量?

2) Is there a way of scaling the number of queue workers running based on the number of jobs waiting to be processed?

问题 2 的原因是我们有一批繁忙的时间，然后是很多安静的时间，所以我真的不想浪费资源让 3 个侦听器一直在运行.

The reason for question 2 is that we have batches of busy times and then lots of quiet times, so i don't really want to be wasting resources with 3 listeners running the whole time.

推荐答案

在 supervisor 中，您使用参数 numprocs 指定进程数量，因此您可以在脚本中添加一行内容:

In supervisor you specify the amount of processes with the parameter numprocs, so you can add a line to your script that says:

<代码>numprocs=5

现在，您可以做一些聪明的事情，例如，如果只有在队列上运行的某些进程花费的时间太长，您可以创建一组不同的队列进程来处理这些进程和其他轻量进程的集合.为了实现这一点，您可以使用一个队列名称(如 --queue=longprocess 和另一个队列名称)创建一个主管配置 --queue=lightprocess 并在您的程序中调度作业在相应的队列中，这样长进程不会延迟短进程.

Now, you can do some clever things, for example if only some of the processes that runs on the queues takes too long, you can create a different set of queues processes to work those and other set for the light processes. To achieve that you can create a supervisor configuration with one queue name like --queue=longprocess and another one with --queue=lightprocess and in your program you dispatch the job in the corresponding queue, that way, long processes won't delay short processes.

您还可以在一个主管配置文件中指定 队列优先级，例如 <代码>--queue=lightprocess,longprocess.这样，您的工作人员将在运行 longprocess 之前首先查找 lightprocess.

You can also specify queue priorities within one supervisor configuration file such as --queue=lightprocess,longprocess. That way, your worker(s) will first look for lightprocess before running longprocess.

回答你的第二个问题，不，就主管而言，所有进程都在运行，它不知道队列是忙还是闲，因此它不能杀死进程并根据它们的使用创建它们，所以没有，您不能动态配置仅在您拥有的进程繁忙时创建更多进程.

To answer your second question, no, in terms of supervisor all processes are running, it doesn't knows if the queue is busy or idle, therefore it can't kill processes and create them based on their use, so no, you can't have a dynamic configuration of creating more processes only when the ones you have are busy.

注意，如果分配超过 1 个 numprocs，则必须将进程数添加到名称中.根据主管配置文档:

A note, if you assign more than 1 numprocs, you must add the number of process to the name. According to supervisor configuration docs:

Supervisor 将启动与 numprocs 命名的程序一样多的实例.请注意，如果 numprocs > 1，则 process_name 表达式必须包含 %(process_num)s(或任何其他包含 process_num 的有效 Python 字符串表达式).

Supervisor will start as many instances of this program as named by numprocs. Note that if numprocs > 1, the process_name expression must include %(process_num)s (or any other valid Python string expression that includes process_num) within it.

主管配置文档:http://supervisord.org/configuration.html

这篇关于使用 Supervisor 运行多个 Laravel 队列工作者的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！