芹菜工人仅在未分离时才导入任务 [英] celery worker only imports tasks when not detached

问题描述

我正在尝试让django应用程序将任务提交给芹菜工作者，并且在该工作者附加运行时成功了.一旦添加--detach，任务就无法注册.

I'm trying to get my django app to submit tasks to a celery worker and it's succeeding when the worker is run attached. As soon as I add the --detach the tasks are failing to be registered.

[2020-10-26 04:09:33,159: ERROR/MainProcess] Received unregistered task of type 'devapp.tasks.create_random_files'.
The message has been ignored and discarded.

Did you remember to import the module containing this task?
Or maybe you're using relative imports?

Please see
http://docs.celeryq.org/en/latest/internals/protocol.html
for more information.

The full contents of the message body was:
'[[20, "blah", 5120], {}, {"callbacks": null, "errbacks": null, "chain": null, "chord": null}]' (93b)
Traceback (most recent call last):
  File "/pysite/project/venv/lib/python3.6/site-packages/celery/worker/consumer/consumer.py", line 562, in on_task_received
    strategy = strategies[type_]
KeyError: 'devapp.tasks.create_random_files'

在我的task.py中，我有

In my tasks.py I have

import os
import string
import subprocess
from celery import shared_task, task

@shared_task(name='devapp.tasks.create_random_files')
def create_random_files(total,filename,size):
    for i in range(int(total)):
        filenum = str(i).zfill(5)
        rnd_filename = '/brdis/{}-{}'.format(filenum,filename)
        with open(rnd_filename, 'wb') as f:
            f.write(os.urandom(size))
            f.close
    return '{} random files created'.format(total)

在我的views.py中，我有

and in my views.py I have

from django.shortcuts import render
from django.http import HttpResponse

from django.contrib import messages
from django.views.generic import TemplateView
from django.views.generic.list import ListView
from django.views.generic.edit import FormView
from django.shortcuts import redirect

from .forms import CreateRandomFilesForm, GetFileListForm, ClamScanFileForm
from .tasks import create_random_files, clamscanfile

class ClamScanFileView(FormView):
    template_name = 'devapp/clamscan_file.html'
    form_class = ClamScanFileForm

    def form_valid(self, form):
        filename = form.cleaned_data.get('filename')
        clamscanfile.delay(filename)
        messages.success(self.request, 'We are scanning your file!s Wait a moment and refresh this page.')
        return redirect('files_list')

我知道这可能是相对进口的(我需要弄清楚一些其他事情)，但是我不明白为什么在芹菜工作者命令中添加--detach会导致此错误:

I understand that it may be relative imports(something else i need to get my head round) but I don't understand why adding the --detach to the celery worker command produces this error:

celery-项目工作人员-E --loglevel = INFO --logfile =/var/log/celery/celeryd.log --pidfile =/var/run/celery/celeryd.pid

工作正常，但是...

works fine but...

celery-项目工作人员-E --loglevel = INFO --logfile =/var/log/celery/celeryd.log --pidfile =/var/run/celery/celeryd.pid --detach

启动工作程序但不注册任务?

starts the worker but doesn't register the tasks?

任何帮助表示赞赏.

FWIW

celery report

software -> celery:4.4.7 (cliffs) kombu:4.6.11 py:3.6.8
            billiard:3.6.3.0 py-amqp:2.6.1
platform -> system:Linux arch:64bit, ELF
            kernel version:3.10.0-1127.19.1.el7.x86_64 imp:CPython
loader   -> celery.loaders.default.Loader
settings -> transport:amqp results:disabled

推荐答案

这是4.4.7的错误.降级到4.4.6或升级到5应该可以解决此问题.

This is a bug with 4.4.7. Downgrading to 4.4.6 or upgrading to 5 should fix it.

https://github.com/celery/celery/issues/6370

这篇关于芹菜工人仅在未分离时才导入任务的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！