Docker-Celery作为守护程序-未找到pidfiles [英] Docker - Celery as a daemon - no pidfiles found

问题描述

我似乎已经尝试过这里的所有解决方案，但是似乎都没有用，我不确定我缺少什么.我试图通过docker容器将celery作为守护程序运行.

I seem to have tried every solution on here but none seem to be working, I'm not sure what I'm missing. Im trying to run celery as a daemon through my docker container.

root@bae5de770400:/itapp/itapp# /etc/init.d/celeryd status
celery init v10.1.
Using config script: /etc/default/celeryd
celeryd down: no pidfiles found
root@bae5de770400:/itapp/itapp# /etc/init.d/celerybeat status
celery init v10.1.
Using configuration: /etc/default/celeryd
celerybeat is down: no pid file found
root@bae5de770400:/itapp/itapp#

我已经看到很多关于烫发的帖子，而我都尝试过，但都无济于事.

I've seen lots of posts to do with perms and I've tried them all to no avail.

这是我的docker文件，可创建所有权限和文件夹

this is my docker file which creates all the perms and folders

FROM python:latest
ENV PYTHONUNBUFFERED 1

# add source for snmp
RUN sed -i "s#jessie main#jessie main contrib non-free#g" /etc/apt/sources.list
# install dependancies
RUN apt-get update -y \
    && apt-get install -y apt-utils python-software-properties libsasl2-dev python3-dev libldap2-dev libssl-dev libsnmp-dev snmp-mibs-downloader git vim

# copy and install requirements
RUN mkdir /config  
ADD /config/requirements.txt /config/  
RUN pip install -r /config/requirements.txt  
# create folders
RUN mkdir /itapp;
RUN mkdir /static;
# create celery user
RUN useradd -N -M --system -s /bin/false celery
RUN echo celery:"*****" | /usr/sbin/chpasswd
# celery perms
RUN groupadd grp_celery
RUN usermod -a -G grp_celery celery
RUN mkdir /var/run/celery/
RUN mkdir /var/log/celery/
RUN chown root:root /var/run/celery/
RUN chown root:root /var/log/celery/
# copy celery daemon files
ADD /config/celery/init_celeryd /etc/init.d/celeryd
RUN chmod +x /etc/init.d/celeryd
ADD /config/celery/celerybeat /etc/init.d/celerybeat
RUN chmod +x /etc/init.d/celerybeat
RUN chmod 755 /etc/init.d/celeryd
RUN chown root:root  /etc/init.d/celeryd
RUN chmod 755 /etc/init.d/celerybeat
RUN chown root:root /etc/init.d/celerybeat
# copy celery config
ADD /config/celery/default_celeryd /etc/default/celeryd
# RUN /etc/init.d/celeryd start
# set workign DIR for copying code
WORKDIR /itapp

如果我手动启动它会起作用

if I start it manually it works

celery -A itapp worker -l info
/usr/local/lib/python3.6/site-packages/celery/platforms.py:795: RuntimeWarning: You're running the worker with superuser privileges: this is
absolutely not recommended!

Please specify a different user using the -u option.
...

[2017-09-25 17:29:51,707: INFO/MainProcess] Connected to amqp://it-app:**@rabbitmq:5672/it-app-vhost
[2017-09-25 17:29:51,730: INFO/MainProcess] mingle: searching for neighbors
[2017-09-25 17:29:52,764: INFO/MainProcess] mingle: all alone

从celery存储库中复制了init.d文件，这是我的默认文件的内容(如果有帮助的话)

the init.d files are copied from the celery repo and this is the contents of my default file if it helps

# Names of nodes to start
#   most people will only start one node:
CELERYD_NODES="worker1"
#   but you can also start multiple and configure settings
#   for each in CELERYD_OPTS
#CELERYD_NODES="worker1 worker2 worker3"
#   alternatively, you can specify the number of nodes to start:
#CELERYD_NODES=10

# Absolute or relative path to the 'celery' command:
CELERY_BIN="/usr/local/bin/celery"

# App instance to use
# comment out this line if you don't use an app
CELERY_APP="itapp"
# or fully qualified:

# Where to chdir at start.
CELERYD_CHDIR="/itapp/itapp/"

# Extra command-line arguments to the worker
CELERYD_OPTS="flower --time-limit=300 --concurrency=8"
# Configure node-specific settings by appending node name to arguments:
#CELERYD_OPTS="--time-limit=300 -c 8 -c:worker2 4 -c:worker3 2 -Ofair:worker1"

# Set logging level to DEBUG
#CELERYD_LOG_LEVEL="DEBUG"

# %n will be replaced with the first part of the nodename.
CELERYD_LOG_FILE="/var/log/celery/%n%I.log"
CELERYD_PID_FILE="/var/run/celery/%n.pid"

# Workers should run as an unprivileged user.
#   You need to create this user manually (or you can choose
#   a user/group combination that already exists (e.g., nobody).
CELERYD_USER="celery"
CELERYD_GROUP="celery"

# If enabled pid and log directories will be created if missing,
# and owned by the userid/group configured.
CELERY_CREATE_DIRS=1

此文件中唯一可能出错的地方是我认为是CELERY_BIN值，我不确定在Docker容器中也要设置什么

the only thing in this file which may be wrong I think is the CELERY_BIN value, I'm not sure what to set that too in a docker container

谢谢

推荐答案

因此您的Dockerfile中几乎没有问题

So you had few issues in your Dockerfile

• Celery进程外壳设置为/bin/false ，该进程不允许启动任何进程.
• 您需要向 celery 用户
授予/var/run/celery 和/var/log/celery 的权限>
• /etc/default/celeryd 应该具有640权限
• Dockerfile中的层太多了

• Celery process shell was set to /bin/false which didn't allow any process to be started.
• You needed to give permission on /var/run/celery and /var/log/celery to the celery user
• /etc/default/celeryd should be 640 permission
• Also too many layers in your Dockerfile

所以我将Dockerfile更新到了下面

So I updated the Dockerfile to below

FROM python:latest
ENV PYTHONUNBUFFERED 1

# add source for snmp
RUN sed -i "s#jessie main#jessie main contrib non-free#g" /etc/apt/sources.list
# install dependancies
RUN apt-get update -y \
    && apt-get install -y apt-utils python-software-properties libsasl2-dev python3-dev libldap2-dev libssl-dev libsnmp-dev git vim

# copy and install requirements
RUN mkdir /config
ADD /config/requirements.txt /config/
RUN pip install -r /config/requirements.txt
# create folders
RUN mkdir /itapp && mkdir /static;
# create celery user
RUN useradd -N -M --system -s /bin/bash celery && echo celery:"B1llyB0n3s" | /usr/sbin/chpasswd
# celery perms
RUN groupadd grp_celery && usermod -a -G grp_celery celery && mkdir -p /var/run/celery/ /var/log/celery/
RUN chown -R celery:grp_celery /var/run/celery/ /var/log/celery/
# copy celery daemon files
ADD /config/celery/init_celeryd /etc/init.d/celeryd
RUN chmod +x /etc/init.d/celeryd
ADD /config/celery/celerybeat /etc/init.d/celerybeat
RUN chmod 750 /etc/init.d/celeryd /etc/init.d/celerybeat
RUN chown root:root  /etc/init.d/celeryd /etc/init.d/celerybeat
# copy celery config
ADD /config/celery/default_celeryd /etc/default/celeryd
RUN chmod 640 /etc/default/celeryd
# set workign DIR for copying code
ADD /itapp/ /itapp/itapp
WORKDIR /itapp

然后进入Web服务容器，一切正常

And then got into the web service container and all worked fine

root@ab658c5d0c67:/itapp/itapp# /etc/init.d/celeryd status
celery init v10.1.
Using config script: /etc/default/celeryd
celeryd down: no pidfiles found
root@ab658c5d0c67:/itapp/itapp# /etc/init.d/celeryd start
celery init v10.1.
Using config script: /etc/default/celeryd
celery multi v4.1.0 (latentcall)
> Starting nodes...
    > worker1@ab658c5d0c67: OK
    > flower@ab658c5d0c67: OK
root@ab658c5d0c67:/itapp/itapp# /etc/init.d/celeryd status
celery init v10.1.
Using config script: /etc/default/celeryd
celeryd down: no pidfiles found
root@ab658c5d0c67:/itapp/itapp# /etc/init.d/celeryd status
celery init v10.1.
Using config script: /etc/default/celeryd
celeryd (node worker1) (pid 66) is up...
root@ab658c5d0c67:/itapp/itapp#

这篇关于Docker-Celery作为守护程序-未找到pidfiles的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！