为什么在使用`python app.py`执行时，我的Flask应用程序能工作，而在使用`heroku local web`或`flask run`时却不能运行? [英] Why does my Flask app work when executing using `python app.py` but not when using `heroku local web` or `flask run`?

问题描述

我写了一个基于Flask的Web应用程序，该应用程序从用户那里获取文本并返回其具有给定分类的可能性(下面是完整脚本).该应用程序会在发出任何请求之前加载一些进行预测所需的训练有素的模型.我目前正在尝试将其部署到Heroku上并遇到一些问题.

I wrote a Flask-based web app that takes text from users and returns the probability that it is of a given classification (full script below). The app loads some of the trained models needed to make predictions before any requests are made. I am currently trying to deploy it on Heroku and experiencing some problems.

执行python ml_app.py时，我可以在本地运行它.但是，当我使用Heroku CLI命令heroku local web尝试在本地运行它以在部署前进行测试时，出现以下错误

I am able to run it locally when I execute python ml_app.py. But when I use the Heroku CLI command heroku local web to try to run it locally to test before deployment, I get the following error

AttributeError:模块'__main__'没有属性'tokenize'

AttributeError: module '__main__' has no attribute 'tokenize'

此错误与在行

tfidf_model = joblib.load('models/tfidf_vectorizer_train.pkl')

我已在脚本顶部导入了所需的功能，以确保正确加载了该功能(from utils import tokenize).假设我可以在使用python ml_app.py的情况下运行它，则此方法有效.但是由于我不知道的原因，当我使用heroku local web时它不会加载.当我尝试在本地运行Flask CLI命令flask run时，它也不起作用.知道为什么吗?

I have imported the required function at the top of the script to ensure that this is loaded properly (from utils import tokenize). This works given that I can run it when I use python ml_app.py. But for reasons I do not know, it doesn't load when I use heroku local web. It also doesn't work when I use the Flask CLI command flask run when trying to run it locally. Any idea why?

我承认我对这里的幕后情况(关于代码的Web开发/部署方面)没有很好的了解，因此任何解释都会有所帮助.

I admit that I do not have a good understanding of what is going on under the hood here (with respect to the web dev./deployment aspect of the code) so any explanation helps.

from flask import Flask, request, render_template
from sklearn.externals import joblib
from utils import tokenize  # custom tokenizer required for tfidf model loaded in load_tfidf_model()

app = Flask(__name__)

models_directory = 'models'


@app.before_first_request
def nbsvm_models():

    global tfidf_model
    global logistic_identity_hate_model
    global logistic_insult_model
    global logistic_obscene_model
    global logistic_severe_toxic_model
    global logistic_threat_model
    global logistic_toxic_model

    tfidf_model = joblib.load('models/tfidf_vectorizer_train.pkl')
    logistic_identity_hate_model = joblib.load('models/logistic_identity_hate.pkl')
    logistic_insult_model = joblib.load('models/logistic_insult.pkl')
    logistic_obscene_model = joblib.load('models/logistic_obscene.pkl')
    logistic_severe_toxic_model = joblib.load('models/logistic_severe_toxic.pkl')
    logistic_threat_model = joblib.load('models/logistic_threat.pkl')
    logistic_toxic_model = joblib.load('models/logistic_toxic.pkl')


@app.route('/')
def my_form():
    return render_template('main.html')


@app.route('/', methods=['POST'])
def my_form_post():
    """
        Takes the comment submitted by the user, apply TFIDF trained vectorizer to it, predict using trained models
    """

    text = request.form['text']

    comment_term_doc = tfidf_model.transform([text])

    dict_preds = {}

    dict_preds['pred_identity_hate'] = logistic_identity_hate_model.predict_proba(comment_term_doc)[:, 1][0]
    dict_preds['pred_insult'] = logistic_insult_model.predict_proba(comment_term_doc)[:, 1][0]
    dict_preds['pred_obscene'] = logistic_obscene_model.predict_proba(comment_term_doc)[:, 1][0]
    dict_preds['pred_severe_toxic'] = logistic_severe_toxic_model.predict_proba(comment_term_doc)[:, 1][0]
    dict_preds['pred_threat'] = logistic_threat_model.predict_proba(comment_term_doc)[:, 1][0]
    dict_preds['pred_toxic'] = logistic_toxic_model.predict_proba(comment_term_doc)[:, 1][0]

    for k in dict_preds:
        perc = dict_preds[k] * 100
        dict_preds[k] = "{0:.2f}%".format(perc)

    return render_template('main.html', text=text,
                           pred_identity_hate=dict_preds['pred_identity_hate'],
                           pred_insult=dict_preds['pred_insult'],
                           pred_obscene=dict_preds['pred_obscene'],
                           pred_severe_toxic=dict_preds['pred_severe_toxic'],
                           pred_threat=dict_preds['pred_threat'],
                           pred_toxic=dict_preds['pred_toxic'])


if __name__ == '__main__':

    app.run(debug=True)

推荐答案

修复了该问题.这是由于我选择了存储在tfidf_vectorizer_train.pkl中的类实例的方式.该模型是在ipython笔记本中创建的，其属性之一取决于我在笔记本中交互定义的令牌生成器功能.我很快了解到，酸洗不能保存类的确切实例，这意味着tfidf_vectorizer_train.pkl不包含我在笔记本中定义的功能.

Fixed it. It was due to the way I picked the class instance stored in tfidf_vectorizer_train.pkl. The model was created in an ipython notebook where one of its attributes depended on a tokenizer function that I defined interactively in the notebook. I soon learned that pickling does not save the exact instance of a class, which means tfidf_vectorizer_train.pkl does not contain the function I defined in the notebook.

要解决此问题，我将tokenizer函数移动到了一个单独的实用程序python文件中，并将该函数导入了我训练过的文件中，然后将模型和进行了腌制.

To fix this, I moved the tokenizer function to a separate utilities python file and imported the function in both the file where I trained and subsequently pickled the model and in the file where I unpickled it.

在代码中，我做到了

from utils import tokenize
...
tfidfvectorizer = TfidfVectorizer(ngram_range=(1, 2), tokenizer=tokenize,
                                      min_df=3, max_df=0.9, strip_accents='unicode',
                                      use_idf=1, smooth_idf=True, sublinear_tf=1)

train_term_doc = tfidfvectorizer.fit_transform(train[COMMENT])

joblib.dump(tfidfvectorizer, 'models/tfidf_vectorizer_train.pkl')
...

在我训练模型的文件中

from utils import tokenize
...
@app.before_first_request
def load_models():
    # from utils import tokenize

    global tfidf_model
    tfidf_model = 
    joblib.load('{}/tfidf_vectorizer_train.pkl'.format(models_directory))
...

包含Web应用程序代码的文件中.

in the file containing the web app code.

这篇关于为什么在使用`python app.py`执行时，我的Flask应用程序能工作，而在使用`heroku local web`或`flask run`时却不能运行?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！