轻松启动多个 rq 工作进程——水平扩展 [英] Start multiple rq worker processes easily ― Horizontal scaling

问题描述

如何在 VPS 中轻松创建大量 rq 工作进程?

How can I create a large number of rq worker processes in a VPS easily?

现在我正在手动打开终端并在其中运行 python3 worker.py，然后重复此操作，直到运行的工作实例数量令人满意为止.我知道这不是一个可扩展的解决方案，所以我怎样才能自动轻松地做到这一点.如果有一些工具可以促进这个过程就好了.

Right now I'm manually opening a terminal and running python3 worker.py in it, and then repeating this until I get a satisfying number of worker instances running. I know this is not a scalable solution, so how can I do it automatically and easily. It'd be nice if there were some tool which facilitates this process.

推荐答案

我最终使用了 Supervisord.这似乎是一个不错的解决方案.

I ended up using Supervisord. It seemed like a pretty good solution.

相关的 supervisord.conf 文件如下所示.

The relevant supervisord.conf file looks like this.

[supervisord]

[program:worker]
command=python worker.py
process_name=%(program_name)s-%(process_num)s
numprocs=20
directory=.
stopsignal=TERM
autostart=true
autorestart=true

将此文件放在与 RQ 工作器脚本 worker.py 相同的目录中后，只需运行以下命令即可生成 20 个 RQ 工作器实例.可以通过 supervisord.conf 文件中的 numprocs 选项指定工作线程的数量.

After putting this file in the same directory as of the RQ worker script worker.py, just running the following command will spawn up 20 instances of the RQ worker. The number of workers can be specified by the numprocs option in the supervisord.conf file.

$ supervisord -n

这篇关于轻松启动多个 rq 工作进程——水平扩展的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！