烧瓶:绕过后台工作者的工作(RQ，Redis) [英] Flask: passing around background worker job (rq, redis)

问题描述

我想做一个非常简单的事情:启动一个工作人员到某个地方，然后将答案返回给用户. 我正在尝试结合使用Flask和RQ.

I want to do a very simple thing: Launch a worker to something and then return the answer to user. I'm trying to do so using a combination of Flask and RQ.

import os
from flask import Flask, session
from somewhere import do_something
from rq import Queue
from worker import conn

app = Flask(__name__)
app.debug = True
app.secret_key = '....'

q = Queue(connection=conn)

@app.route('/make/')
def make():
    job = q.enqueue(do_something, 'argument')
    session['job'] = job
    return 'Done'

@app.route('/get/')
def get():
    try:
        session['job'].refresh()
        out = str(session['job'].result)
    except:
        out = 'No result yet'
    return out

在这个非常简单的示例中，想法是人们去/make/并开始工作.一段时间后，可以转到/get/，工作人员的结果将在那里打印.

The idea in this very simple example is that people go to /make/ and the job starts. After some time there can go to /get/ and the result from the worker will be printed there.

但是一行引起问题:

session['job'] = job

似乎无法腌制该作业，这显然是由Flaks会话使用的.我遇到了错误:

It seems the job cannot be pickled, which is apparently used by the Flaks session. I'm getting the error:

...
10:52:16 web.1     |   File "/Users/julius/twitter-sentiment/venv/lib/python2.7/site-packages/flask/app.py", line 804, in save_session
10:52:16 web.1     |     return self.session_interface.save_session(self, session, response)
10:52:16 web.1     |   File "/Users/julius/twitter-sentiment/venv/lib/python2.7/site-packages/flask/sessions.py", line 205, in save_session
10:52:16 web.1     |     secure=secure, domain=domain)
10:52:16 web.1     |   File "/Users/julius/twitter-sentiment/venv/lib/python2.7/site-packages/werkzeug/contrib/securecookie.py", line 329, in save_cookie
10:52:16 web.1     |     data = self.serialize(session_expires or expires)
10:52:16 web.1     |   File "/Users/julius/twitter-sentiment/venv/lib/python2.7/site-packages/werkzeug/contrib/securecookie.py", line 235, in serialize
10:52:16 web.1     |     self.quote(value)
10:52:16 web.1     |   File "/Users/julius/twitter-sentiment/venv/lib/python2.7/site-packages/werkzeug/contrib/securecookie.py", line 192, in quote
10:52:16 web.1     |     value = cls.serialization_method.dumps(value)
10:52:16 web.1     |   File "/Users/julius/twitter-sentiment/venv/bin/../lib/python2.7/copy_reg.py", line 70, in _reduce_ex
10:52:16 web.1     |     raise TypeError, "can't pickle %s objects" % base.__name__
10:52:16 web.1     | TypeError: can't pickle function objects

我真的希望能有所帮助.我可能以完全错误的方式执行此操作(通过会话传递作业)，但是我不知道还有其他方法可以访问作业的结果...

I really hope something can help. I might be doing this in a completely wrong way (with passing the job via a session), but I have no idea how else to access the result of the job...

我们将不胜感激任何帮助.

Any help will be highly appreciated.

谢谢.

推荐答案

我以前没有使用过rq，但是我看到一个作业具有.key属性.将散列存储在会话中可能会更容易.然后，您可以使用Job类的.fetch方法，该方法本身将调用.refresh()并将作业退还给您.读取.result()将会为您提供工作的当前状态.

I've not used rq before but I see that a job has a .key property. It might be easier to store that hash in your session. Then you can use the Job class's .fetch method which will itself call a .refresh() and return the job to you. Reading the .result() at that point would give you the job's current status.

也许是这样(未经测试):

Maybe like this (untested):

from rq.job import Job

@app.route('/make/')
def make():
    job = q.enqueue(do_something, 'argument')
    session['job'] = job.key
    return 'Done'

@app.route('/get/')
def get():
    try:
        job = Job()
        job.fetch(session['job'])
        out = str(job.result)
    except:
        out = 'No result yet'
    return out

这篇关于烧瓶:绕过后台工作者的工作(RQ，Redis)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！