Keras 功能 API:将 CNN 模型与 RNN 相结合以查看图像序列 [英] Keras functional API: Combine CNN model with a RNN to to look at sequences of images
问题描述
所以我遇到了一个问题,即如何在 Keras 中将 CNN 与 RNN 结合起来.在发布问题时,有人指出这是解决问题的正确方法.显然我只是忽略了原始代码中的一些内容,这让我回答了我自己的问题.
So i was stuck with a question on how to combine a CNN with a RNN in Keras. While posting the question someone pointed me out that this is the correct way to approach the problem. Apparently i just overlooked something in the original code, which made me answer my own question.
原来的问题如下:
如何在 Keras 中创建一个模型,将图像序列作为输入,CNN查看"每个单独的图像,并将 CNN 输出序列输入 RNN?
How do you create a model in Keras that has sequences of images as the input, with a CNN 'looking' at each individual image and the sequence of the CNN output being fed into a RNN?
为了更清楚:
模型一:查看单个图像的 CNN.
模型二:在模型一的 CNN 输出序列上的 RNN.
Model one: a CNN that looks at single images.
Model two: a RNN that at the sequences of the output of the CNN from model one.
例如,CNN 应该看到 5 张图像,并且 CNN 的这 5 个输出序列应该被传递给 RNN.
So for example the CNN should see 5 images and this sequence of 5 outputs from the CNN should be passed on to the RNN.
输入数据格式如下:
(number_of_images, width, height, channels) = (4000, 120, 60, 1)
The input data is in the following format:
(number_of_images, width, height, channels) = (4000, 120, 60, 1)
推荐答案
本题答案如下.
采用这个过度简化的 CNN 模型:
Take this oversimplified CNN model:
cnn = Sequential()
cnn.add(Conv2D(16, (50, 50), input_shape=(120, 60, 1)))
cnn.add(Conv2D(16, (40, 40)))
cnn.add(Flatten()) # Not sure if this if the proper way to do this.
然后就是这个简单的RNN模型:
Then there is this simple RNN model:
rnn = Sequential()
rnn = GRU(64, return_sequences=False, input_shape=(120, 60))
应该连接到密集网络:
dense = Sequential()
dense.add(Dense(128))
dense.add(Dense(64))
dense.add(Dense(1)) # Model output
请注意,为了便于阅读,激活函数等已被省略.
Notice that activation functions and such have been left out for readability.
现在剩下的就是结合这 3 个主要模型.
Now all that is left is combining these 3 main models.
main_input = Input(shape=(5, 120, 60, 1)) # Data has been reshaped to (800, 5, 120, 60, 1)
model = TimeDistributed(cnn)(main_input) # this should make the cnn 'run' 5 times?
model = rnn(model) # combine timedistributed cnn with rnn
model = dense(model) # add dense
最后
final_model = Model(inputs=main_input, outputs=model)
final_model.compile...
final_model.fit...
这篇关于Keras 功能 API:将 CNN 模型与 RNN 相结合以查看图像序列的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!