在Flask App中通过线程使用Keras模型 [英] Using Keras Model in Flask App with Threading

问题描述

我有3个有效的Keras模型，每个模型我都保存了2个文件 structure.json (保存模型结构)和 weight.h5 .

I have 3 working Keras models and each of those I saved it with 2 file structure.json (saving Model Structure) and weight.h5.

我已经构建了Flask应用来加载这些模型.但目前我无法像这样将runing = True添加到run()的选项中:

I've built a Flask App to load these model. But currently I can't add threading=True to option of run() like this:

api.run(threaded=True)

所以我只能使用:

api.run()

在此烧瓶应用程序中，我有两个不同的api(1个GET和1个POST).因为它在一个线程上运行，所以它工作太慢.我的系统每秒有100多个连接，每个连接我都必须加载另一个模型.

In this flask app I have two different api (1 GET and 1 POST). Because it runs on one thread so it works too slowly. My system has over 100 connections by seconds, and each connection I have to load another Model.

请注意，我所有的keras模型都具有相同的结构.我只需要加载一个结构，每当一个新的连接到来时，我就会对该结构加载重量.

我在api中的代码如下:

My code in api likes this:

# Compile model
json_file = open(formal_model_path, 'r')
loaded_model_json = json_file.read()
json_file.close()
model = model_from_json(loaded_model_json)
model.compile(loss='mean_absolute_error', optimizer='adam')

@api.route("/predict", methods=["POST"])
    try:
        model.load_weights(os.path.join(cointainer_folder, 'weight.h5'))
    except Exception:
        return jsonify(
            error_code='model_file_reading_failed'
        )

我的代码在第 model.load_weights [...] 行启用threading = True时出错.

My code ran errors when enabling threading=True at line model.load_weights[...].

是否有任何解决方案可以通过加载许多不同的Keras模型来构建多线程API?

Are there any solutions to build multithreading API with loading many different Keras model?

推荐答案

我认为您遇到了两个单独的问题:

I think you are running into two separate problems:

1. 您正在按请求加载模型权重.这是一个坏主意，会使每个请求非常缓慢.

1. You are loading model weights per request. That's a bad idea and will make every request very slow.

flask使用多个线程.在同一线程中必须使用在一个线程中加载的Tensorflow模型.

flask uses multiple threads. Tensorflow models loaded in one thread, must be used in that same thread.

加载模型的正确位置是init方法.您还需要使用tf.get_default_graph()来确保要在同一线程中加载模型并进行预测.

The right place to load models would be an init method. You also need to use tf.get_default_graph() to make sure that you are loading models and predicting in the same thread.

这就是您的代码的样子

def init():
    global models
    models = {}
    for idx, model_path in enumerate(model_paths):
        with open(model_path, "r") as fp:
            model = model_from_json(json.load(fp))
            model.compile(loss='mean_absolute_error', optimizer='adam')
            model.load_weights(os.path.join(model_path, "weights.h5"))
            models[idx] = model

        # save default graph in a global var
        global graph
        graph = tf.get_default_graph()

在您的请求处理程序中

@api.route("/predict", methods=["POST"])
def predict():
    # select your model based on something inside the request
    # making up func !
    model = models[func(request)]

    with graph.as_default():
        model.predict(..)

这篇关于在Flask App中通过线程使用Keras模型的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！