如何在django中设置celeryconfig文件 [英] How to set celeryconfig file in in django

问题描述

我有在Ubuntu EC2节点上的Django项目，我一直在使用 Celery 来设置异步。我一直在努力追随 http://michal.karzynski.pl/blog/2014/05/18/setting-up-an-asynchronous-task-queue-for-django-using-celery-redis/

我已经能够在命令行中使用以下工具获取基本任务：

 （env1）ubuntu @ ip-172-31-22-65：〜/ projects / tp $ celery --app = tp.celery：app worker --loglevel = INFO 
 
（env1）ubuntu @ ip-172-31-22-65：〜/ projects / tp $ celery --app = tp.celery：app worker --loglevel = INFO 
 
 --------------芹菜@ ip-172-31-22-65 v3.1.17（Cipater）
 ---- **** ----- 
 --- * *** *  -  Linux-3.13.0-44-generic-x86_64-with-Ubuntu-14.04-trusty 
  -  *  -  **** --- 
  -  ** ---------- [config] 
  -  ** ----------。> app：tp：0x7f66a89c0470 
  -  ** ----------。>运输：redis：// localhost：6379/0 
  -  ** ----------。>结果：disabled 
  -  *** --- * ---。>并发性：1（prefork）
  -  ******* ---- 
 --- ***** ----- [queues] 
 --- -----------。>芹菜交换=芹菜（直接）键=芹菜
  

但是，如果我运行其他芹菜命令，如下所示获取以下内容：

 （env1）ubuntu @ ip-172-31-22-65：〜/ projects / tp $ celery worker [2015-04-03 13：17：21,553：WARNING / MainProcess] /home/ubuntu/.virtualenvs/env1/lib/python3.4/site-packages/celery/apps/worker.py:161：
 
 
 --------------芹菜@ ip-172-31-22-65 v3.1.17（Cipater）
 ---- **** ----- 
 --- * *** *  -  Linux-3.13.0-44-generic-x86_64-with-Ubuntu-14.04-trusty 
  -  *  - **** --- 
  -  ** ---------- [config] 
  -  ** ----------。> app：default：0x7f1653eae7b8（.default.Loader）
  -  ** ----------。>运输：amqp：// guest：** @ localhost：5672 // 
  -  ** ----------。>结果：disabled 
  -  *** --- * ---。>并发性：1（prefork）
  -  ******* ---- 
 --- ***** ----- [queues] 
 --- -----------。>芹菜交换=芹菜（直接）键=芹菜
 
 
 [2015-04-03 13：17：21,571：错误/ MainProcess]消费者：无法连接到amqp：// guest：* *@127.0.0.1：5672 //：[Errno 111]拒绝连接。 
  

似乎芹菜认为我使用amqp作为一个经纪人，但我正在使用redis！ ！

根据 Celery尝试连接到错误的代理，似乎芹菜找不到配置文件并使用默认值。

在上述问题中推荐：

 导入您的芹菜，并添加您的经纪人：
 
芹菜=芹菜'，broker ='redis：//127.0.0.1：6379'）
 celery.config_from_object（celeryconfig）
  

我该怎么做？是我的celery.py文件（以下），与芹菜配置相同？

/projects/tp/tp/celery.py

  from __future__ import absolute_import 
 
 import os 
 import django 
从芹菜进口芹菜
从django .conf导入设置
 
＃设置'芹菜'程序的默认Django设置模块。 
 os.environ.setdefault（'DJANGO_SETTINGS_MODULE'，'tp.settings'）
 django.setup（）
 
 app = Celery（'hello_django'）
 
＃在这里使用一个字符串表示工作人员在使用Windows时不必使用
＃pickle该对象。 
 app.config_from_object（'django.conf：settings'）
 app.autodiscover_tasks（lambda：settings.INSTALLED_APPS）
  

tasks.py：

  from __future__ import absolute_import 
 from celery import shared_task 
从django.core.cache导入缓存
 
 
 @shared_task 
 def tester1（param）：
 return'使用参数执行的测试任务％s '％param 
  

tp / tp1 / views

$ b tester1.delay（'hi'）

返回HttpResponse（请求）：

（'hTML'）

/etc/supervisor/conf.d/tp-celery.conf

  [程序：tp-celery] 
 command = / home / ubuntu / .virtualenvs / env1 / bin / celery  - app = tp.celery：app worker --loglevel = INFO 
 directory = / home / ubuntu / projects / tp 
 user = ubuntu 
 numprocs = 1 
 stdout_logfile = / var / log / celery-worker-out.log 
 stderr_logfile = / var / log /celery-worker-err.log 
 
 autostart = true 
 autorestart = true 
 startsecs = 10 
 
;需要等待正在执行的任务在关机时完成。 
;如果你有很长的运行任务，可以增加这个。 
 stopwaitsecs = 600 
 
;当诉诸发送SIGKILL程序来终止它
;将SIGKILL发送到其整个流程组，而
;也照顾孩子。 
 killasgroup = true 
  

/var/log/celery-worker-out.log

  --------------芹菜@ ip-172-31-22-65 v3.1.17 （Cipater）
 ---- **** ----- 
 --- * *** *  -  Linux-3.13.0-44-generic-x86_64-with-Ubuntu- 14.04-trusty 
  -  *  -  **** --- 
  -  ** ---------- [config] 
  -  ** ----- -----。> app：tp：0x7fa33e424cf8 
  -  ** ----------。>运输：redis：// localhost：6379/0 
  -  ** ----------。>结果：disabled 
  -  *** --- * ---。>并发性：1（prefork）
  -  ******* ---- 
 --- ***** ----- [queues] 
 --- -----------。>芹菜交换=芹菜（直接）键=芹菜
 
 
 [任务] 
。 testapp.tasks.tester1 
  

解决方案

不要运行芹菜woker only..run like 芹菜-A tp worker -l info 。这将需要默认的 config 。

对于芹菜检查

 芹菜--app = tp.celery：app inspect active_queues 
  

或简单

 芹菜 - a tp inspect active_queues 
  

I have a Django project on an Ubuntu EC2 node, which I have been using to set up an asynchronous using Celery. I've been trying to follow http://michal.karzynski.pl/blog/2014/05/18/setting-up-an-asynchronous-task-queue-for-django-using-celery-redis/

I've been able to get a basic task working at the command line, using:

(env1)ubuntu@ip-172-31-22-65:~/projects/tp$ celery --app=tp.celery:app worker --loglevel=INFO

(env1)ubuntu@ip-172-31-22-65:~/projects/tp$ celery --app=tp.celery:app worker --loglevel=INFO

 -------------- celery@ip-172-31-22-65 v3.1.17 (Cipater)
---- **** -----
--- * ***  * -- Linux-3.13.0-44-generic-x86_64-with-Ubuntu-14.04-trusty
-- * - **** ---
- ** ---------- [config]
- ** ---------- .> app:         tp:0x7f66a89c0470
- ** ---------- .> transport:   redis://localhost:6379/0
- ** ---------- .> results:     disabled
- *** --- * --- .> concurrency: 1 (prefork)
-- ******* ----
--- ***** ----- [queues]
 -------------- .> celery           exchange=celery(direct) key=celery

However, if I run other celery commands like below I'm getting the following:

(env1)ubuntu@ip-172-31-22-65:~/projects/tp$ celery worker                       [2015-04-03 13:17:21,553: WARNING/MainProcess] /home/ubuntu/.virtualenvs/env1/lib/python3.4/site-packages/celery/apps/worker.py:161: 


 -------------- celery@ip-172-31-22-65 v3.1.17 (Cipater)
---- **** -----
--- * ***  * -- Linux-3.13.0-44-generic-x86_64-with-Ubuntu-14.04-trusty
-- * - **** ---
- ** ---------- [config]
- ** ---------- .> app:         default:0x7f1653eae7b8 (.default.Loader)
- ** ---------- .> transport:   amqp://guest:**@localhost:5672//
- ** ---------- .> results:     disabled
- *** --- * --- .> concurrency: 1 (prefork)
-- ******* ----
--- ***** ----- [queues]
 -------------- .> celery           exchange=celery(direct) key=celery


[2015-04-03 13:17:21,571: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**@127.0.0.1:5672//: [Errno 111] Connection refused.

It appears that celery thinks I'm using amqp as a broker , but I'm using redis!!

Based on Celery tries to connect to the wrong broker, it seems likely that celery cannot find the configuration file and uses the default.

In the above question they recommend:

import your celery and add your broker like that : 

celery = Celery('task', broker='redis://127.0.0.1:6379')
celery.config_from_object(celeryconfig)

Where would I do this? is my celery.py file (below), the same as a celery config?

/projects/tp/tp/celery.py

from __future__ import absolute_import

import os
import django
from celery import Celery
from django.conf import settings

# set the default Django settings module for the 'celery' program.
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'tp.settings')
django.setup()

app = Celery('hello_django')

# Using a string here means the worker will not have to
# pickle the object when using Windows.
app.config_from_object('django.conf:settings')
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)

tasks.py:

from __future__ import absolute_import
from celery import shared_task
from django.core.cache import cache


@shared_task
def tester1(param):
    return 'The test task executed with argument "%s" ' % param

tp/tp1/views

@csrf_exempt
def tester(request):

    tester1.delay('hi')

    return HttpResponse('hTML')

/etc/supervisor/conf.d/tp-celery.conf

[program:tp-celery]
command=/home/ubuntu/.virtualenvs/env1/bin/celery --app=tp.celery:app worker --loglevel=INFO
directory=/home/ubuntu/projects/tp
user=ubuntu
numprocs=1
stdout_logfile=/var/log/celery-worker-out.log
stderr_logfile=/var/log/celery-worker-err.log

autostart=true
autorestart=true
startsecs=10

; Need to wait for currently executing tasks to finish at shutdown.
; Increase this if you have very long running tasks.
stopwaitsecs = 600

; When resorting to send SIGKILL to the program to terminate it
; send SIGKILL to its whole process group instead,
; taking care of its children as well.
killasgroup=true

/var/log/celery-worker-out.log

 -------------- celery@ip-172-31-22-65 v3.1.17 (Cipater)
---- **** ----- 
--- * ***  * -- Linux-3.13.0-44-generic-x86_64-with-Ubuntu-14.04-trusty
-- * - **** --- 
- ** ---------- [config]
- ** ---------- .> app:         tp:0x7fa33e424cf8
- ** ---------- .> transport:   redis://localhost:6379/0
- ** ---------- .> results:     disabled
- *** --- * --- .> concurrency: 1 (prefork)
-- ******* ---- 
--- ***** ----- [queues]
 -------------- .> celery           exchange=celery(direct) key=celery


[tasks]
  . testapp.tasks.tester1

解决方案

Dont run celery woker only..run like celery -A tp worker -l info. It will take default config.

For celery inspect

celery --app=tp.celery:app inspect active_queues

OR simply

celery -A tp inspect active_queues

这篇关于如何在django中设置celeryconfig文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！