celerybeat-多个实例&监控 [英] celerybeat - multiple instances & monitoring

问题描述

我正在使用celery构建应用程序，最近我们需要按计划运行某些任务.

I'm having application built using celery and recently we got a requirement to run certain tasks on schedule.

我认为celerybeat非常适合此操作，但是我有几个问题:

I think celerybeat is perfect for this, but I got few questions:

1. 是否可以运行多个celerybeat实例，以使任务不重复?
2. 如何确保celerybeat始终启动&跑步?

到目前为止，我读到以下内容: https://github.com/celery/celery/issues/251 和 https://github.com/ybrs/single-beat

So far I read this: https://github.com/celery/celery/issues/251 and https://github.com/ybrs/single-beat

似乎应该运行一个celerybeat实例.

It looks like a single instance of celerybeat should be running.

我正在AWS Elasticbeanstalk docker容器内运行应用程序，而celery worker也是docker容器(因此在需要时可以快速扩展).

I'm running application inside AWS elasticbeanstalk docker containers and celery workers are also docker containers (so it's quickly scaleable when needed).

最好让芹菜节拍和芹菜工人一起经过监督员，但这似乎不是正确的方法.

It would be best to have celerybeat run through supervisord along with celery workers, but it seems this is not proper way to do this.

同时具有celerybeat的单个实例将需要手动配置/启动和监视.

At the same time having that single instance of celerybeat would require manual provision/start and monitoring.

推荐答案

要回答您的2个问题:

1. 如果您运行多个celerybeat实例，则会得到重复的任务，因此afaik您应该只有一个celerybeat实例.

1. If you run several celerybeat instances you get duplicated tasks, so afaik you should have only single celerybeat instance.

我使用的是 supervisord ，它可以将芹菜工人和celebeatbeat工人作为守护进程来运行，因此它们应始终处于启动状态.正在运行.

I'm using supervisord as you mentioned to run celery workers and celerybeat workers as deamon so they should always be up & running.

我的监督配置:

[program:my_regular_worker]
command=python2.7 /home/ubuntu/workspace/src/manage.py celery worker -Q my_regular_worker-queue_name -c 1 -l info --without-mingle
process_name=my_regular_worker
directory=/home/ubuntu/workspace/src
autostart=true
autorestart=true
user=ubuntu
stdout_logfile=/tmp/my_regular_worker.log
redirect_stderr=true



[program:my_celerybeat_worker]
command=python2.7 /home/ubuntu/workspace/src/manage.py celery worker -Q my_celerybeat_worker-queue_name -c 1 -l info --without-mingle -B -s /tmp/celerybeat-schedule

这篇关于celerybeat-多个实例&监控的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！