keras模型加载极慢 [英] Extremely slow model load with keras
问题描述
我有一套Keras模型(30),我使用以下方法进行了训练和保存:
I have a set of Keras models (30) that I trained and saved using:
model.save('model{0}.h5'.format(n_model))
当我尝试使用load_model
加载它们时,每个模型所需的时间都非常大且需要增量.加载过程如下:
When I try to load them, using load_model
, the time required for each model is quite large and incremental. The loading is done as:
models = {}
for i in range(30):
start = time.time()
models[i] = load_model('model{0}.h5'.format(ix))
end = time.time()
print "Model {0}: seconds {1}".format(ix, end - start)
输出为:
...
Model 9: seconds 7.38966012001
Model 10: seconds 9.99283003807
Model 11: seconds 9.7262301445
Model 12: seconds 9.17000102997
Model 13: seconds 10.1657290459
Model 14: seconds 12.5914049149
Model 15: seconds 11.652477026
Model 16: seconds 12.0126030445
Model 17: seconds 14.3402299881
Model 18: seconds 14.3761711121
...
每个模型都非常简单:2个隐藏层,每个都有10个神经元(大小约50Kb).为什么装载量如此之大,为什么时间会增加?我是否缺少某些内容(例如模型的关闭功能?)
Each model is really simple: 2 hidden layers with 10 neurons each (size ~50Kb). Why is the loading taking so much and why is the time increasing? Am I missing something (e.g. close function for the model?)
解决方案
我发现,加快模型的加载最好将网络的结构和权重存储到两个不同的文件中: 保存部分:
I found out that to speed up the loading of the model is better to store the structure of the networks and the weights into two distinct files: The saving part:
model.save_weights('model.h5')
model_json = model.to_json()
with open('model.json', "w") as json_file:
json_file.write(model_json)
json_file.close()
加载部分:
from keras.models import model_from_json
json_file = open("model.json", 'r')
loaded_model_json = json_file.read()
json_file.close()
model = model_from_json(loaded_model_json)
model.load_weights("model.h5")
推荐答案
我通过在每次加载之前清除keras会话来解决了这个问题
I solved the problem by clearing the keras session before each load
from keras import backend as K
for i in range(...):
K.clear_session()
model = load_model(...)
这篇关于keras模型加载极慢的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!