具有时间分布的Keras预训练CNN [英] Keras pretrain CNN with TimeDistributed

问题描述

这是我的问题，我想在TimeDistributed层中使用一种预训练的CNN网络.但是我在实施它时遇到了一些问题.

Here is my problem, I want to use one of the pretrain CNN network in a TimeDistributed layer. But I have some problem to implement it.

这是我的模特:

def bnn_model(max_len):
    # sequence length and resnet input size
    x = Input(shape=(maxlen, 224, 224, 3))

    base_model = ResNet50.ResNet50(weights='imagenet',  include_top=False)

    for layer in base_model.layers:
        layer.trainable = False

    som = TimeDistributed(base_model)(x)

    #the ouput of the model is [1, 1, 2048], need to squeeze
    som = Lambda(lambda x: K.squeeze(K.squeeze(x,2),2))(som)

    bnn = Bidirectional(LSTM(300))(som)
    bnn = Dropout(0.5)(bnn)

    pred = Dense(1, activation='sigmoid')(bnn)

    model = Model(input=x, output=pred)

    model.compile(optimizer=Adam(lr=1.0e-5), loss="mse", metrics=["accuracy"])

    return model

编译模型时，我没有错误.但是当我开始训练时，出现以下错误:

When compiling the model I have no error. But when I start training I get the following error:

tensorflow/core/framework/op_kernel.cc:975] Invalid argument: You must feed a value for placeholder tensor 'input_2' with dtype float
[[Node: input_2 = Placeholder[dtype=DT_FLOAT, shape=[], _device="/job:localhost/replica:0/task:0/gpu:0"]()]]

我检查了并确实发送了float32，但是对于input1，input2是pretrain Resnet中存在的输入.

I checked and I do send float32 but for input1, input2 is the input present in the pretrain Resnet.

这里只是概述，是模型摘要. (注意:很奇怪，它没有显示Resnet内部发生的事情，但是没关系)

Just to have an overview here is the model summary. (Note: it's strange that it doesn't show what happen inside Resnet but never mind)

____________________________________________________________________________________________________
Layer (type)                     Output Shape          Param #     Connected to                     
====================================================================================================
input_1 (InputLayer)             (None, 179, 224, 224, 0                                            
____________________________________________________________________________________________________
timedistributed_1 (TimeDistribut (None, 179, 1, 1, 204 23587712    input_1[0][0]                    
____________________________________________________________________________________________________
lambda_1 (Lambda)                (None, 179, 2048)     0           timedistributed_1[0][0]          
____________________________________________________________________________________________________
bidirectional_1 (Bidirectional)  (None, 600)           5637600     lambda_1[0][0]                   
____________________________________________________________________________________________________
dropout_1 (Dropout)              (None, 600)           0           bidirectional_1[0][0]            
____________________________________________________________________________________________________
dense_1 (Dense)                  (None, 1)             601         dropout_1[0][0]                  
====================================================================================================
Total params: 29,225,913
Trainable params: 5,638,201
Non-trainable params: 23,587,712
____________________________________________________________________________________________________

我猜我没有正确使用TimeDistributed，我发现没有人尝试这样做.我希望有人可以指导我.

I am guessing that I do not use the TimeDistributed correctly and I saw nobody trying to do this. I hope someone can guide me on this.

问题来自于ResNet50.ResNet50(weights='imagenet', include_top=False)在图形中创建自己的输入的事实.

The problem comes from the fact that ResNet50.ResNet50(weights='imagenet', include_top=False) create its own input in the graph.

所以我想我需要做类似ResNet50.ResNet50(weights='imagenet', input_tensor=x, include_top=False)的事情，但是我看不到如何将其与TimeDistributed结合.

So I guess I need to do something like ResNet50.ResNet50(weights='imagenet', input_tensor=x, include_top=False) but I do not see how to couple it with TimeDistributed.

我尝试过

base_model = Lambda(lambda x : ResNet50.ResNet50(weights='imagenet',  input_tensor=x, include_top=False))
som = TimeDistributed(base_model)(in_ten)

但是它不起作用.

推荐答案

我的快速解决方案有点难看.

My quick solution is a little bit ugly.

我只是复制了ResNet的代码，并将TimeDistributed添加到所有层，然后将基本" ResNet的权重加载到了自定义的ResNet上.

I just copied the code of ResNet and added TimeDistributed to all layers and then loaded the weights from a "basic" ResNet on my customized ResNet.

注意:

要能够像这样分析图像序列，确实会在GPU上占用大量内存.

To be able to analyze sequence of images like this does take a huge amount of memory on the gpu.

这篇关于具有时间分布的Keras预训练CNN的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！