如何在Docker容器（Python，Flask和Redis）中启动自定义RQ工作者 [英] How to start a custom RQ worker within a Docker Container (Python, Flask and Redis)

问题描述

我遵循了Miguel Grinberg出色的Flask Mega教程，并成功地在Docker容器中安装了带有Redis任务队列和RQ工作者的Flask Web应用。

为了提高任务队列性能，我现在需要使用自己的自定义工作程序，而不是默认的RQ工作程序。

不幸的是，我正在努力了解如何在docker中启动自定义工作器。

要启动一个作为默认的RQ工作者，Flask Mega教程使用的方法是使用 venv / bin / rq覆盖Docker入口点，然后提供参数 worker -u redis：// redis-server：6379/0 microblog-tasks。 / p>

可执行文件的名称带有--entrypoint标志，而命令参数则在容器映像名称之后的末尾传递。

这是完整的命令-仅最后两行与此问题相关。

  $ docker run --name rq-worker -d --rm -e SECRET_KEY =我的秘密密钥\ 
 -e MAIL_SERVER = smtp.googlemail.com -e MAIL_PORT = 587 -e MAIL_USE_TLS = true ＼ ＼ 
 -e MAIL_USERNAME =<您的Gmail用户名> -e MAIL_PASSWORD =<您的Gmail密码> \ 
 --link mysql：dbserver --link redis：redis-server \ 
 -e DATABASE_URL = mysql + pymysql：// microblog：<数据库密码> @ dbserver / microblog \ 
 -e REDIS_URL = redis：// redis-server：6379/0 \ 
 --entrypoint venv / bin / rq \ 
 microblog：latest worker -u redis：// redis -server：6379/0 microblog-tasks 
  

我有自己的自定义工作程序，其代码如下直接从RQ文档中获取：

 ＃！/ usr / bin / env python 
 import sys 
 from rq import Connection，Worker 
 
＃预加载库
 import library_that_you_want_preloaded 
 
＃提供队列名称作为此脚本的参数来监听，
＃与带有Connection（）的rq worker 
：
 qs = sys.argv [1：]或['default'] 
 
w = Worker（qs）
 w.work （）
  

鉴于我的自定义工作人员位于 home / dashboard / app / custom_worker.py，我该执行哪些命令在使用我的自定义工作程序脚本启动Docker容器以创建RQ工作程序时需要提供什么？到目前为止，我已经尝试了以下操作：

  $ docker run --name rq-worker -d --rm -e SECRET_KEY = my-secret-key \ 
 -e MAIL_SERVER = smtp.googlemail.com -e MAIL_PORT = 587 -e MAIL_USE_TLS = true \ 
 -e MAIL_USERNAME =<您的Gmail用户名> -e MAIL_PASSWORD =<您的Gmail密码> \ 
 --link mysql：dbserver --link redis：redis-server \ 
 -e DATABASE_URL = mysql + pymysql：// microblog：<数据库密码> @ dbserver / microblog \ 
 -e REDIS_URL = redis：// redis-server：6379/0 \ 
 --entrypoint venv / bin / rq \ 
 microblog：latest / home / dashboard / app / custom_worker .py -u redis：// redis-server：6379/0 microblog-tasks 
  

。 ..

  $ docker run --name rq-worker -d --rm -e SECRET_KEY =我的秘密密钥＼ ＼ 
 -e MAIL_SERVER = smtp.googlemail.com -e MAIL_PORT = 587 -e MAIL_USE_TLS = true \ 
 -e MAIL_USERNAME =<您的Gmail用户名> -e MAIL_PASSWORD =<您的Gmail密码> \ 
 --link mysql：dbserver --link redis：redis-server \ 
 -e DATABASE_URL = mysql + pymysql：// microblog：<数据库密码> @ dbserver / microblog \ 
 -e REDIS_URL = redis：// redis-server：6379/0 \ 
 --entrypoint / home / dashboard / app \ 
 microblog：latest custom_worker -u redis：// redis-server：6379/0 microblog-tasks 
  

任何帮助将不胜感激。在线上有很多关于创建自定义RQ工作程序的帖子，但是我还没有找到有关如何在部署中实际使用自定义工作程序的详细信息。

谢谢友善的，
罗宾

解决方案

在文档中可能使用以下命令：

/ usr / local / bin / rq worker -w custom_worker.py --path path / to / sourcecode

使用/ usr / local / bin / rq worker查看更多选项-帮助

文档：
https://python-rq.org/docs/workers/#custom-worker-classes

I followed the excellent Flask Mega Tutorial by Miguel Grinberg and have successfully setup a Flask web app with a Redis task queue and RQ workers, all in Docker containers.

To improve task queue performance, I now need to use my own custom worker, rather than the default RQ worker.

Unfortunately, I'm struggling to understand how I start a custom worker within docker.

To start a default RQ worker, the Flask Mega Tutorial uses the method of overriding the Docker entrypoint with "venv/bin/rq" and then supplying the argument "worker -u redis://redis-server:6379/0 microblog-tasks".

The executable name is supplied with the --entrypoint flag, whilst the command arguments are passed at the very end, after the name of the container image.

Here is the full command - only the last two lines are relevant to this question.

$ docker run --name rq-worker -d --rm -e SECRET_KEY=my-secret-key \
-e MAIL_SERVER=smtp.googlemail.com -e MAIL_PORT=587 -e MAIL_USE_TLS=true \
-e MAIL_USERNAME=<your-gmail-username> -e MAIL_PASSWORD=<your-gmail-password> \
--link mysql:dbserver --link redis:redis-server \
-e DATABASE_URL=mysql+pymysql://microblog:<database-password>@dbserver/microblog \
-e REDIS_URL=redis://redis-server:6379/0 \
--entrypoint venv/bin/rq \
microblog:latest worker -u redis://redis-server:6379/0 microblog-tasks

I have my own custom worker with the following code, taken directly from the RQ documentation:

#!/usr/bin/env python
import sys
from rq import Connection, Worker

# Preload libraries
import library_that_you_want_preloaded

# Provide queue names to listen to as arguments to this script,
# similar to rq worker
with Connection():
    qs = sys.argv[1:] or ['default']

    w = Worker(qs)
    w.work()

Given that my custom worker is located within the Docker container at "home/dashboard/app/custom_worker.py", which commands do I need to supply upon starting the Docker container to create an RQ worker using my customised worker script? So far I have tried the following:

$ docker run --name rq-worker -d --rm -e SECRET_KEY=my-secret-key \
-e MAIL_SERVER=smtp.googlemail.com -e MAIL_PORT=587 -e MAIL_USE_TLS=true \
-e MAIL_USERNAME=<your-gmail-username> -e MAIL_PASSWORD=<your-gmail-password> \
--link mysql:dbserver --link redis:redis-server \
-e DATABASE_URL=mysql+pymysql://microblog:<database-password>@dbserver/microblog \
-e REDIS_URL=redis://redis-server:6379/0 \
--entrypoint venv/bin/rq \
microblog:latest /home/dashboard/app/custom_worker.py -u redis://redis-server:6379/0 microblog-tasks

and also...

$ docker run --name rq-worker -d --rm -e SECRET_KEY=my-secret-key \
-e MAIL_SERVER=smtp.googlemail.com -e MAIL_PORT=587 -e MAIL_USE_TLS=true \
-e MAIL_USERNAME=<your-gmail-username> -e MAIL_PASSWORD=<your-gmail-password> \
--link mysql:dbserver --link redis:redis-server \
-e DATABASE_URL=mysql+pymysql://microblog:<database-password>@dbserver/microblog \
-e REDIS_URL=redis://redis-server:6379/0 \
--entrypoint /home/dashboard/app \
microblog:latest custom_worker -u redis://redis-server:6379/0 microblog-tasks

Any help would be greatly appreciated. There are a lot of posts online about creating a custom RQ worker, but I've not found much detail on how you practically use your custom worker in deployment.

Thank you kindly, Robin

解决方案

In the docs its possible with below command:

/usr/local/bin/rq worker -w custom_worker.py --path path/to/sourcecode

See more options with /usr/local/bin/rq worker --help

Docs: https://python-rq.org/docs/workers/#custom-worker-classes

这篇关于如何在Docker容器（Python，Flask和Redis）中启动自定义RQ工作者的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！