Redis 后台作业完成后如何返回flask render_template? [英] How do I return flask render_template after the Redis background job is done?
问题描述
我在 Flask 中有这个 Web 应用程序，我想在提交表单后在其中执行一些 ML 和 AI 算法.我在 Redis 和 rq 的帮助下在后台作业中运行 ML 和 AI 算法(因为我的应用程序由 Heroku 托管，并且他们有超时问题，您必须在 30 秒内返回响应).工作完成后，我想获取算法制作的图像(一些图形)并将它们输出到网页中，但我不知道如何在工作功能中呈现模板，以及如何从烧瓶中导入应用程序执行此操作的应用程序似乎不起作用.您对如何解决这个问题有什么想法吗?
I have this web app in a flask where I want to execute some ML and AI algorithms after a form is submitted. I am running the ML and AI algorithms in a background job with the help of Redis and rq ( because I have my app hosted by Heroku and they have this timeout thing where you have to return a response within 30 seconds). After the job is done I would like to get the images made by the algorithms ( some graphs ) and output them in a web page, but I have no idea how to render a template in a job function, and importing the app from the flask app to do that doesn't seem to work. Do you have any ideas on how to solve this?
来自将作业排入队列的 Flask 应用程序的代码片段:
my code fragment from the flask app that enqueues a job:
def upload():
from mlsalespred import run_model
file = request.files['file']
dffile = pd.read_csv(file)
job = q.enqueue(run_model, dffile)
return render_template("waiting.html")
我的工作职能代码片段:
my code fragment from the job function:
def run_model(dataFrame):
- - - - - - - - - - -
- - some ml stuff - -
- - - - - - - - - - -
return render_template("uploaded.html", sales_fig = sales_fig.decode('utf8'), diff_fig = diff_fig.decode('utf8'), pred_fig = pred_fig.decode('utf8') )
提前致谢
推荐答案
一个基本但可行的解决方案 (gist):
您可以通过从将作业排入队列的路由重定向来实现此目的，然后让元标记定期刷新该页面.首先导入所需的库:
A basic but workable solution (gist):
You could do this by just redirecting from the route which enqueues the job, then have a meta tag refresh that page periodically. First import the required libraries:
from flask import Flask, redirect, url_for, render_template_string
app = Flask(__name__)
from time import sleep
from rq import Queue
from rq.job import Job
from redis import Redis
设置rq相关连接，并定义要运行的函数:
Set up the rq related connections, and define the function to run:
r = Redis(host='redisserver')
q = Queue(connection=r)
def slow_func(data):
sleep(5)
return 'Processed %s' % (data,)
然后定义一个可以每5秒刷新一次页面的模板:
Then define a template which can refresh the page every 5 seconds:
template_str='''<html>
<head>
{% if refresh %}
<meta http-equiv="refresh" content="5">
{% endif %}
</head>
<body>{{result}}</body>
</html>'''
我们还将使用flask render_template_string 创建一个辅助函数来返回插入了变量的模板.请注意，如果未提供，refresh 默认为 False:
We'll also make a helper function to return that template with a variable inserted, using flask render_template_string. Notice that refresh defaults to False, if not supplied:
def get_template(data, refresh=False):
return render_template_string(template_str, result=data, refresh=refresh)
现在创建一个路由，将我们的函数加入队列，获取其 rq 作业 ID，然后使用该 id 返回到 result 视图的重定向.这只是在 URL 字符串中输入，但可以从任何地方获取:
Now make a route which will enqueue our function, get its rq job-id, then return a redirect to the result view with that id. This just takes input in the URL string, but could get that from anywhere:
@app.route('/process/<string:data>')
def process(data):
job = q.enqueue(slow_func, data)
return redirect(url_for('result', id=job.id))
现在让我们在 rq.Job 对象的帮助下处理实际结果.可以调整此处的逻辑，因为这将导致对除 finished" 之外的所有值进行页面刷新:
Now let's handle the actual result, with the help of the rq.Job object. The logic here could be tweaked, as this will cause a page refresh on all values except "finished":
@app.route('/result/<string:id>')
def result(id):
job = Job.fetch(id, connection=r)
status = job.get_status()
if status in ['queued', 'started', 'deferred', 'failed']:
return get_template(status, refresh=True)
elif status == 'finished':
result = job.result
# If this is a string, we can simply return it:
return get_template(result)
如果状态为 finished" 那么 job.result 将包含 slow_func 的返回值，所以我们在页面.
If the status is "finished" then job.result will contain the return value of slow_func, so we render this on the page.
这种方法的缺点是在等待作业完成时会向服务器发出多个请求.元刷新标签可能有点不合常规.如果您从 Javascript 发送更新请求，那么有解决方案可以每隔一段时间发送一次 AJAX 请求，尽管这遇到了相同的多请求问题.
This method has the disadvantage of causing several requests to the server, whilst waiting for job completion. The meta refresh tag may be a bit unconventional. If you're sending the request for an update from Javascript, then there are solutions which can send the AJAX request at an interval, though this suffers from the same multiple request problem.
另一种方法是使用 websockets 或 SSE 将已完成作业的结果在完成后立即传输到前端.
The alternative is to use websockets, or SSE to stream the result of the completed job to the frontend as soon as it completes.
更新:2021 年 2 月 27 日
我决定尝试使用 SSE 方法更新前端的工作状态.我了解到 rq 通过在作业中导入 rq.get_current_job 来支持更新作业中的 meta 属性，然后可以作业刷新后外部访问.
I decided to have a go at the SSE method of updating the frontend with job status. I learned that rq has native support for updating a meta attribute within the job, by importing rq.get_current_job inside the job, which can then be accessed externally after job refresh.
查看演示代码:
这篇关于Redis 后台作业完成后如何返回flask render_template?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
