Flask中的Celery任务，用于上传和调整图像大小并将其存储到Amazon S3 [英] Celery task in Flask for uploading and resizing images and storing it to Amazon S3

问题描述

我正在尝试创建一个celery任务，用于在将图像存储到Amazon S3之前对其进行上载和调整大小.但是它没有按预期工作.没有任务，一切工作正常.到目前为止，这是代码:

I'm trying to create a celery task for uploading and resizing an image before storing it to Amazon S3. But it doesn't work as expected. Without the task everything is working fine. This is the code so far:

stacktrace

stacktrace

Traceback (most recent call last):
  File "../myVE/lib/python2.7/site-packages/kombu/messaging.py", line 579, in _receive_callback
    decoded = None if on_m else message.decode()
  File "../myVE/lib/python2.7/site-packages/kombu/transport/base.py", line 147, in decode
    self.content_encoding, accept=self.accept)
  File "../myVE/lib/python2.7/site-packages/kombu/serialization.py", line 187, in decode
    return decode(data)
  File "../myVE/lib/python2.7/site-packages/kombu/serialization.py", line 74, in pickle_loads
    return load(BytesIO(s))
  File "../myVE/lib/python2.7/site-packages/werkzeug/datastructures.py", line 2595, in __getattr__
    return getattr(self.stream, name)
  File "../myVE/lib/python2.7/site-packages/werkzeug/datastructures.py", line 2595, in __getattr__
    return getattr(self.stream, name)
    ...
RuntimeError: maximum recursion depth exceeded while calling a Python object

views.py

from PIL import Image

from flask import Blueprint, redirect, render_template, request, url_for

from myapplication.forms import UploadForm
from myapplication.tasks import upload_task


main = Blueprint('main', __name__)

@main.route('/upload', methods=['GET', 'POST'])
def upload():
    form = UploadForm()
    if form.validate_on_submit():
        upload_task.delay(form.title.data, form.description.data,
                          Image.open(request.files['image']))
        return redirect(url_for('main.index'))
    return render_template('upload.html', form=form)

tasks.py

from StringIO import StringIO

from flask import current_app

from myapplication.extensions import celery, db
from myapplication.helpers import resize, s3_upload
from myapplication.models import MyObject


@celery.task(name='tasks.upload_task')
def upload_task(title, description, source):
    stream = StringIO()
    target = resize(source, current_app.config['SIZE'])
    target.save(stream, 'JPEG', quality=95)
    stream.seek(0)
    obj = MyObject(title=title, description=description, url=s3_upload(stream))
    db.session.add(obj)
    db.session.commit()

推荐答案

您似乎正在尝试将整个上传的文件作为Celery消息的一部分传递.我想这会给您带来麻烦.我建议您查看是否可以将文件作为视图的一部分保存到Web服务器，然后让消息(延迟"参数)包含文件名而不是整个文件的数据.然后，该任务可以从硬盘驱动器中读取文件，上传到s3，然后在本地将其删除.

It looks like you are attempting to pass the entire uploaded file as part of the Celery message. I imagine that is causing you some trouble. I would recommend seeing if you can save the file to the web server as part of the view, then have the message (the "delay" argument) contain the filename rather than entire file's data. The task can then read the file in from the hard drive, upload to s3, then delete it locally.

这篇关于Flask中的Celery任务，用于上传和调整图像大小并将其存储到Amazon S3的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！