在GPU上与keras集成预测 [英] Ensemble forecast with keras on GPU
问题描述
我正在尝试使用同样构建的keras模型进行整体预测.单个NN的数据具有相同的形状.我想使用GPU，因为应该并行训练模型.因此，我正在尝试合并模型.因为模型的数量应该是可配置的，所以我想循环进行.我找到了一些解决方案，但是对于每个循环我都有麻烦.这是我的方法:
I am trying to make an ensemble forecast with equally build keras models. The Data for the single NNs are of the same shape. I want to use the GPU because the models should be trained parallel. Therefor I am trying to merge the models. Because the number of models should be configurable, I want to do this in a loop. I found some solutions, but for each I have troubles with the loop. Here is my approach:
from keras import Sequential, Model
from keras.layers import Embedding, GlobalAveragePooling1D, Dense, concatenate
import numpy as np
nummodels=3
model = Sequential()
model.add(Embedding(20, 10, trainable=True))
model.add(GlobalAveragePooling1D())
model.add(Dense(1, activation='sigmoid'))
for i in range(nummodels-1):
model2 = Sequential()
model2.add(Embedding(20, 10, trainable=True))
model2.add(GlobalAveragePooling1D())
model2.add(Dense(1, activation='sigmoid'))
model_concat = concatenate([model.output, model2.output], axis=-1)
model_concat = Dense(1, activation='softmax')(model_concat)
model = Model(inputs=[model.input, model2.input], outputs=model_concat)
model.compile(loss='binary_crossentropy', optimizer='adam')
# somehow generating testdata x1,x2,x3 and y1,y2,y3...
# not implemented yet...
# Training
model.fit([x1,x2,x3],[y1,y2,y3], epochs = 50)
# prediction
ypred1,ypred2,ypred3 = model.predict([x1,x2,x3])
循环不起作用.我希望你能告诉我是什么问题.稍后，我还将循环训练和预测.我的代码中的拟合和预测是否正确?如果不可能循环执行此操作，请给我其他解决方案.
The loop is not working. I hope you can tell me what's the problem. Later on I will train and predict also in a loop. Is the fitting and prediction correct in my code? If it's not possible to do this in a loop, please give me other solutions.
基于M.Innat的回答，我更改了代码.现在，在循环中，我将NN的输入和输出添加到一个列表中，稍后我将使用该列表进行装饰.但是在第一个循环之后的恢复仍然无法正常工作.
Based on the Answer from M.Innat, I changed my code. Now, in the loop I add the inputs and outputs of the NNs to a list which I use lateron for concaternation. But the concaternation after the first loop is still not working.
from keras import Model
from keras.layers import Dense, Input
import numpy as np
import tensorflow as tf
nummodels=3
inputs=[]
outputs=[]
final_outputs=[]
init = 'uniform'
activation = 'tanh'
for i in range(nummodels):
input_layer = Input(shape=(11,))
A2 = Dense(8, kernel_initializer=init, activation=activation)(input_layer)
A2 = Dense(5, kernel_initializer=init, activation=activation)(A2)
A2 = Dense(3, kernel_initializer=init, activation=activation)(A2)
A2 = Dense(1, kernel_initializer=init, activation=activation)(A2)
model = Model(inputs=input_layer, outputs=A2, name="Model"+str(i+1))
inputs.append(model.input)
outputs.append(model.output)
model_concat = tf.keras.layers.concatenate(outputs, name='target_concatenate')
for i in range(nummodels):
final_outputs.append(Dense(1, activation='sigmoid')(model_concat))
# whole models
composed_model = tf.keras.Model(inputs=inputs, outputs=final_outputs)
# model viz
tf.keras.utils.plot_model(composed_model)
# compile the model
composed_model.compile(loss='binary_crossentropy', optimizer='adam')
# data generation
x1 = np.random.randint(1000, size=(32, 11))
x2 = np.random.randint(1000, size=(32, 11))
x3 = np.random.randint(1000, size=(32, 11))
y1 = np.random.randint(1000, size=(32, 1))
y2 = np.random.randint(1000, size=(32, 1))
y3 = np.random.randint(1000, size=(32, 1))
# Training
composed_model.fit([x1,x2,x3],[y1,y2,y3], epochs = 50)
# prediction
y1p, y2p, y3p = composed_model.predict([x1,x2,x3])
推荐答案
首先，您尝试构建 nummodels 乘以模型并连接其输出.其次，您尝试使用多输入对模型进行 .fit 并尝试获得多输出-也就是说，您的每个模型都会分别输入，合并后的模型将给出 nummodels 次输出.因此，根据我对您问题的理解，这是您的一种可能的解决方案.
Firstly, you are trying to build nummodels times models and concate their output. And secondly, you try to .fit the model with multi-input and try to get multi-output - that means, your every model would take separately inputs and the merged model would give nummodels times output. So, based on my understanding of your question here is one possible solution for you.
模型
import tensorflow as tf
from tensorflow.keras import Input, Model
from tensorflow.keras.layers import Dense, Embedding, GlobalAveragePooling1D
# Define Model A
input_layer = Input(shape=(10,))
A2 = Embedding(1000, 64)(input_layer)
A2 = GlobalAveragePooling1D()(A2)
model_a = Model(inputs=input_layer, outputs=A2, name="ModelA")
# Define Model B
input_layer = Input(shape=(10,))
A2 = Embedding(1000, 64)(input_layer)
A2 = GlobalAveragePooling1D()(A2)
model_b = Model(inputs=input_layer, outputs=A2, name="ModelB")
# Define Model C
input_layer = Input(shape=(10,))
A2 = Embedding(1000, 64)(input_layer)
A2 = GlobalAveragePooling1D()(A2)
model_c = Model(inputs=input_layer, outputs=A2, name="ModelC")
合并模型
import numpy as np
import tensorflow as tf
# concate their output
model_concat = tf.keras.layers.concatenate(
[
model_a.output, model_b.output, model_c.output
], name='target_concatenate')
# final activation layer
final_out1 = Dense(1, activation='sigmoid')(model_concat)
final_out2 = Dense(1, activation='sigmoid')(model_concat)
final_out3 = Dense(1, activation='sigmoid')(model_concat)
# whole models
composed_model = tf.keras.Model(
inputs=[model_a.input, model_b.input, model_c.input],
outputs=[final_out1, final_out2, final_out3]
)
# model viz
tf.keras.utils.plot_model(composed_model)
I/O
# compile the model
composed_model.compile(loss='binary_crossentropy', optimizer='adam')
# dummies
input_array1 = np.random.randint(1000, size=(32, 10))
input_array2 = np.random.randint(1000, size=(32, 10))
input_array3 = np.random.randint(1000, size=(32, 10))
# get some prediction - multi-input / multi-output
output_array1, output_array2, output_array3 = composed_model.predict([input_array1,
input_array2,
input_array3])
检查这些答案以获得更多理解.
Check these answers for more understanding.
根据您的评论和问题的编辑部分，尝试如下操作:
Based on your comment and edited part of your question, try as follows:
models = []
for i in range(nummodels):
....
....
model = Model(inputs=input_layer, outputs=A2, name="Model"+str(i+1))
models.append(model)
# inputs.append(model.input)
# outputs.append(model.output)
# model_concat = tf.keras.layers.concatenate(outputs, name='target_concatenate')
model_concat = tf.keras.layers.concatenate(
[
models[0].output, models[1].output, models[2].output
], name='target_concatenate')
final_outputs=[]
for i in range(nummodels):
final_outputs.append(Dense(1, activation='sigmoid')(model_concat))
# whole models
composed_model = tf.keras.Model(inputs=[models[0].input,
models[1].input,
models[2].input],
outputs=final_outputs)
这篇关于在GPU上与keras集成预测的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
