Flask中的代码503与嵌入式散景服务器应用程序通过requests.get（）获取jsonified数据 [英] Code 503 in Flask with Embedded Bokeh Server App fetching jsonified data through requests.get()

问题描述

我正在通过使Flask应用程序通过专用于通过查询字符串参数传递所请求数据的路径来公开模型数据来参数化我的散景应用程序。我知道数据发送路由的作品，因为当我使用它作为一个url AjaxDataSource 我得到预期的数据绘制。但是，当我尝试使用 requests.get api等价操作时，我得到一个503响应代码，这让我觉得我违反了一些基本的东西，我不能完全理解我有限的webdev经验。我在做什么错误和/或违反？

实际上我需要更多的数据检索灵活性比 AjaxDataSource 提供与列的限制。我希望依靠请求模块来传递任意的类实例，以及通过序列化和反序列化Json来传递什么。

<这里是我展示了从

从屏幕截图可以看出，直接从 sendModelData 适当地呈现JSon，但是当通过 requests.get 方法获取时，由于在Python中报告了503代码控制台。

如果我试图消除代理，我已经通过环境变量启用了这个方法，但是这个方法从来没有完成，并且请求让浏览器无限期地旋转。

想想看，甚至把请求作为中间人使用也是完全没有必要的，我应该能够得到json字符串并自己进行反序列化。那么，这将工作在这个设置中，在我的实际代码中，Bokeh渲染是在一个完全不同的python模块中完成的，因此这些函数甚至不可用，除非我争夺应用程序的分层。
$ b

编辑
事实证明，我违反的基本原则是Flask的开发环境...

lockquote

您正在使用Flask测试服务器运行WSGI应用程序，
默认使用单个线程来处理请求。所以，当你的一个
请求线程试图回调到同一个服务器时，仍然是
忙于处理这个请求。
https://stackoverflow.com/a/22878916/1330381

那么问题就变成了如何在原始的Bokeh例子中应用这个threaded = True技术？这可能是由于flask_embed.py示例对Tornado WSGI服务器的依赖，这个服务器从这个问题建议Tornado是单线程的设计。
鉴于上述发现，一个更加令人关切的问题是， AjaxDataSource 如何共同避免请求所面临的这些线程问题 module？

I'm in the process of parameterizing my bokeh apps by having my Flask app expose model data via a route dedicated to jsonifying the requested data passed via query string arguments. I know the data sending route works since when I use it as a url to AjaxDataSource I get the expected data plotted. However when I attempt the equivalent operation using the requests.get api I get a 503 response code which makes me think I'm violating something fundamental here I can't quite grasp with my limited webdev experience. What am I doing wrong and or violating?

I actually need a bit more data retrieving flexibility than the AjaxDataSource provides with its columnar limitations. I was hoping to lean on the requests module to pass arbitrary class instances and what not around by serializing and deserializing Json.

Here's the minimal example I have demonstrating the failure derived from flask_embed.html...

import requests
from flask import Flask, jsonify, render_template
import numpy as np
import pandas
from tornado.ioloop import IOLoop

from bokeh.application          import Application
from bokeh.application.handlers import FunctionHandler
from bokeh.embed                import autoload_server
from bokeh.layouts              import column
from bokeh.models               import AjaxDataSource,ColumnDataSource
from bokeh.plotting             import figure
from bokeh.server.server        import Server

import pdb

flask_app = Flask(__name__)

# Populate some model maintained by the flask application
modelDf = pandas.DataFrame()
nData = 100
modelDf[ 'c1_x' ] = range(nData)
modelDf[ 'c1_y' ] = [ x*x for x in range(nData) ]
modelDf[ 'c2_x' ] = range(nData)
modelDf[ 'c2_y' ] = [ 2*x for x in range(nData) ]

def modify_doc1(doc):
    # get colum name from query string
    args      = doc.session_context.request.arguments
    paramName = str( args['colName'][0].decode('utf-8') )

    # get model data from Flask
    url    = "http://localhost:8080/sendModelData/%s" % paramName 
    source = AjaxDataSource( data             = dict( x=[] , y=[] ) ,
                             data_url         = url       ,
                             polling_interval = 5000      ,
                             mode             = 'replace' ,
                             method           = 'GET'     )
    # plot the model data
    plot = figure( )
    plot.circle( 'x' , 'y' , source=source , size=2 )
    doc.add_root(column(plot))

def modify_doc2(doc):
    # get column name from query string
    args    = doc.session_context.request.arguments
    colName = str( args['colName'][0].decode('utf-8') )

    # get model data from Flask
    url = "http://localhost:8080/sendModelData/%s" % colName
    pdb.set_trace()
    res = requests.get( url , timeout=None , verify=False )
    print( "CODE %s" % res.status_code )
    print( "ENCODING %s" % res.encoding )
    print( "TEXT %s" % res.text )
    data = res.json()

    # plot the model data
    plot = figure()
    plot.circle( 'x' , 'y' , source=data , size=2 )
    doc.add_root(column(plot))


bokeh_app1 = Application(FunctionHandler(modify_doc1))
bokeh_app2 = Application(FunctionHandler(modify_doc2))

io_loop = IOLoop.current()

server = Server({'/bkapp1': bokeh_app1 , '/bkapp2' : bokeh_app2 }, io_loop=io_loop, allow_websocket_origin=["localhost:8080"])
server.start()

@flask_app.route('/', methods=['GET'] )
def index():
    res =  "<table>"
    res += "<tr><td><a href=\"http://localhost:8080/app1/c1\">APP1 C1</a></td></tr>"
    res += "<tr><td><a href=\"http://localhost:8080/app1/c2\">APP1 C2</a></td></tr>"
    res += "<tr><td><a href=\"http://localhost:8080/app2/c1\">APP2 C1</a></td></tr>"
    res += "<tr><td><a href=\"http://localhost:8080/app2/c2\">APP2 C2</a></td></tr>"
    res += "<tr><td><a href=\"http://localhost:8080/sendModelData/c1\">DATA C1</a></td></tr>"
    res += "<tr><td><a href=\"http://localhost:8080/sendModelData/c2\">DATA C2</a></td></tr>"
    res += "</table>"
    return res

@flask_app.route( '/app1/<colName>' , methods=['GET'] )
def bkapp1_page( colName ) :
    script = autoload_server( model=None , url='http://localhost:5006/bkapp1' )
    script = appendQuery( script , "colName" , colName )
    return render_template("embed.html", script=script)

@flask_app.route( '/app2/<colName>' , methods=['GET'] )
def bkapp2_page( colName ) :
    script = autoload_server( model=None , url='http://localhost:5006/bkapp2' )
    script = appendQuery( script , "colName" , colName )
    return render_template("embed.html", script=script)

@flask_app.route('/sendModelData/<colName>' , methods=['GET'] )
def sendModelData( colName ) :
    x = modelDf[ colName + "_x" ].tolist()
    y = modelDf[ colName + "_y" ].tolist()
    return jsonify( x=x , y=y )

def appendQuery( script , key , value) :
    # Pass along the parameter as a query string to the script's src url: TODO this will formally be introduced in next release of Bokeh to avoid this hack
    script_list = script.split("\n")
    idxSrcAttr = 2
    script_list[idxSrcAttr] = script_list[idxSrcAttr][:-1] + "&{}={}\"".format( key , value )
    script = "\n".join(script_list)
    return script

if __name__ == '__main__':
    from tornado.httpserver import HTTPServer
    from tornado.wsgi import WSGIContainer
    from bokeh.util.browser import view

    print('Opening Flask app with embedded Bokeh application on http://localhost:8080/')

    # This uses Tornado to server the WSGI app that flask provides. Presumably the IOLoop
    # could also be started in a thread, and Flask could server its own app directly
    http_server = HTTPServer(WSGIContainer(flask_app))
    http_server.listen(8080)

    io_loop.add_callback(view, "http://localhost:8080/")
    io_loop.start()

Here's the pages rendered...

Here's some debug output...

C:\TestApp>python flask_embedJSONRoute.py
Opening Flask app with embedded Bokeh application on http://localhost:8080/
> C:\TestApp\flask_embedjsonroute.py(52)modify_doc2()
-> res = requests.get( url , timeout=None , verify=False )
(Pdb) n
> C:\TestApp\flask_embedjsonroute.py(53)modify_doc2()
-> print( "CODE %s" % res.status_code )
(Pdb) n
CODE 503
> C:\TestApp\flask_embedjsonroute.py(54)modify_doc2()
-> print( "ENCODING %s" % res.encoding )
(Pdb) n
ENCODING utf-8
> C:\TestApp\flask_embedjsonroute.py(55)modify_doc2()
-> print( "TEXT %s" % res.text )
(Pdb) n
TEXT
> C:\TestApp\flask_embedjsonroute.py(56)modify_doc2()
-> data = res.json()
(Pdb)

  File "C:\Anaconda3\lib\json\decoder.py", line 357, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

解决方案

This appears to be not an issue with Bokeh per se but rather an issue with threading and blocking in the server that's running the Flask app.

It's reproducible apart from Bokeh entirely...

import requests
from flask import Flask, jsonify, request
import pandas
import pdb

flask_app = Flask(__name__)

# Populate some model maintained by the flask application
modelDf = pandas.DataFrame()
nData = 100
modelDf[ 'c1_x' ] = range(nData)
modelDf[ 'c1_y' ] = [ x*x for x in range(nData) ]
modelDf[ 'c2_x' ] = range(nData)
modelDf[ 'c2_y' ] = [ 2*x for x in range(nData) ]

@flask_app.route('/', methods=['GET'] )
def index():
    res =  "<table>"
    res += "<tr><td><a href=\"http://localhost:8080/sendModelData/c1\">SEND C1</a></td></tr>"
    res += "<tr><td><a href=\"http://localhost:8080/sendModelData/c2\">SEND C2</a></td></tr>"
    res += "<tr><td><a href=\"http://localhost:8080/RequestsOverFlaskNoProxy?colName=c1\">REQUEST OVER FLASK NO PROXY C1</a></td></tr>"
    res += "<tr><td><a href=\"http://localhost:8080/RequestsOverFlaskNoProxy?colName=c2\">REQUEST OVER FLASK NO PROXY C2</a></td></tr>"
    res += "<tr><td><a href=\"http://localhost:8080/RequestsOverFlask?colName=c1\">REQUEST OVER FLASK C1</a></td></tr>"
    res += "<tr><td><a href=\"http://localhost:8080/RequestsOverFlask?colName=c2\">REQUEST OVER FLASK C2</a></td></tr>"
    res += "</table>"   
    return res

@flask_app.route('/RequestsOverFlaskNoProxy')
def requestsOverFlaskNoProxy() :
    print("RequestsOverFlaskNoProxy")
    # get column name from query string
    colName = request.args.get('colName')

    # get model data from Flask
    url = "http://localhost:8080/sendModelData/%s" % colName

    print("Get data from %s" % url )
    session = requests.Session()
    session.trust_env = False
    res = session.get( url , timeout=5000 , verify=False )
    print( "CODE %s" % res.status_code )
    print( "ENCODING %s" % res.encoding )
    print( "TEXT %s" % res.text )
    data = res.json()
    return data

@flask_app.route('/RequestsOverFlask')
def requestsOverFlask() :
    # get column name from query string
    colName = request.args.get('colName')

    # get model data from Flask
    url = "http://localhost:8080/sendModelData/%s" % colName
    res = requests.get( url , timeout=None , verify=False )
    print( "CODE %s" % res.status_code )
    print( "ENCODING %s" % res.encoding )
    print( "TEXT %s" % res.text )
    data = res.json()
    return data

@flask_app.route('/sendModelData/<colName>' , methods=['GET'] )
def sendModelData( colName ) :
    x = modelDf[ colName + "_x" ].tolist()
    y = modelDf[ colName + "_y" ].tolist()
    return jsonify( x=x , y=y )

if __name__ == '__main__':
    print('Opening Flask app on http://localhost:8080/')

    # THIS DOES NOT WORK
    #flask_app.run( host='0.0.0.0' , port=8080 , debug=True )

    # THIS WORKS
    flask_app.run( host='0.0.0.0' , port=8080 , debug=True , threaded=True ) 

One can see from the screen shot that serving data directly from sendModelData renders the JSon appropriately, but when fetched via the requests.get method yields an exception due to a 503 code as reported in the Python Console.

If I make the same attempt trying to eliminate the effect of the proxies which I have enabled via environment variables but this approach never completes and the request leaves the browser spinning indefinitely.

Come to think of it it may be completely unnecessary to even use requests as a middle man and I should be able to just get the json string and go about deserializing it myself. Well, that would work in this setup by in my actual code the Bokeh rendering is done in a completely different python Module than the Flask application so these functions are not even available unless I scramble the layering of the app.

EDIT As it turns out the fundamental thing I was violating was with Flask's development environment...

You are running your WSGI app with the Flask test server, which by default uses a single thread to handle requests. So when your one request thread tries to call back into the same server, it is still busy trying to handle that one request. https://stackoverflow.com/a/22878916/1330381

So then the question becomes how to apply this threaded=True technique in the original Bokeh example? This may not be possible by the flask_embed.py example's reliance on the Tornado WSGI server which from this question suggests Tornado is single threaded by design. Given the above findings an even keener question is how does the AjaxDataSource all together avoid these threading issues faced by the requests module?

这篇关于Flask中的代码503与嵌入式散景服务器应用程序通过requests.get（）获取jsonified数据的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！