Keras VGG16微调 [英] Keras VGG16 fine tuning

问题描述

在

There is an example of VGG16 fine-tuning on keras blog, but I can't reproduce it.

更准确地说，这是用于在没有顶层的情况下初始化VGG16并冻结除最顶层以外的所有块的代码:

More precisely, here is code used to init VGG16 without top layer and to freeze all blocks except the topmost:

WEIGHTS_PATH_NO_TOP = 'https://github.com/fchollet/deep-learning-models/releases/download/v0.1/vgg16_weights_tf_dim_ordering_tf_kernels_notop.h5'
weights_path = get_file('vgg16_weights.h5', WEIGHTS_PATH_NO_TOP)

model = Sequential()
model.add(InputLayer(input_shape=(150, 150, 3)))
model.add(Conv2D(64, (3, 3), activation='relu', padding='same'))
model.add(Conv2D(64, (3, 3), activation='relu', padding='same'))
model.add(MaxPooling2D((2, 2), strides=(2, 2)))

model.add(Conv2D(128, (3, 3), activation='relu', padding='same'))
model.add(Conv2D(128, (3, 3), activation='relu', padding='same'))
model.add(MaxPooling2D((2, 2), strides=(2, 2)))

model.add(Conv2D(256, (3, 3), activation='relu', padding='same'))
model.add(Conv2D(256, (3, 3), activation='relu', padding='same'))
model.add(Conv2D(256, (3, 3), activation='relu', padding='same'))
model.add(MaxPooling2D((2, 2), strides=(2, 2)))

model.add(Conv2D(512, (3, 3), activation='relu', padding='same'))
model.add(Conv2D(512, (3, 3), activation='relu', padding='same'))
model.add(Conv2D(512, (3, 3), activation='relu', padding='same'))
model.add(MaxPooling2D((2, 2), strides=(2, 2)))

model.add(Conv2D(512, (3, 3), activation='relu', padding='same', name='block5_conv1'))
model.add(Conv2D(512, (3, 3), activation='relu', padding='same', name='block5_conv2'))
model.add(Conv2D(512, (3, 3), activation='relu', padding='same', name='block5_conv3'))
model.add(MaxPooling2D((2, 2), strides=(2, 2), name='block5_maxpool'))

model.load_weights(weights_path)

for layer in model.layers:
    layer.trainable = False

for layer in model.layers[-4:]:
    layer.trainable = True
    print("Layer '%s' is trainable" % layer.name)  

接下来，创建一个具有单个隐藏层的顶级模型:

Next, creating a top model with single hidden layer:

top_model = Sequential()
top_model.add(Flatten(input_shape=model.output_shape[1:]))
top_model.add(Dense(256, activation='relu'))
top_model.add(Dropout(0.5))
top_model.add(Dense(1, activation='sigmoid'))
top_model.load_weights('top_model.h5')

请注意，以前曾对瓶颈功能进行过培训，如博客文章中所述.接下来，将此顶级模型添加到基本模型并进行编译:

Note that it was previously trained on bottleneck features like it is described in the blog post. Next, add this top model to the base model and compile:

model.add(top_model)
model.compile(loss='binary_crossentropy',
              optimizer=SGD(lr=1e-4, momentum=0.9),
              metrics=['accuracy'])

并最终适合猫/狗数据:

And eventually, fit on cats/dogs data:

batch_size = 16

train_datagen = ImageDataGenerator(rescale=1./255,
                                   shear_range=0.2,
                                   zoom_range=0.2,
                                   horizontal_flip=True)

test_datagen = ImageDataGenerator(rescale=1./255)

train_gen = train_datagen.flow_from_directory(
    TRAIN_DIR,
    target_size=(150, 150),
    batch_size=batch_size,
    class_mode='binary')

valid_gen = test_datagen.flow_from_directory(
    VALID_DIR,
    target_size=(150, 150),
    batch_size=batch_size,
    class_mode='binary')

model.fit_generator(
    train_gen,
    steps_per_epoch=nb_train_samples // batch_size,
    epochs=nb_epoch,
    validation_data=valid_gen,
    validation_steps=nb_valid_samples // batch_size)

但这是我尝试适应时遇到的错误:

But here is an error I am getting when trying to fit:

ValueError:检查模型目标时出错:预期block5_maxpool具有4个维度，但数组的形状为(16，1)

ValueError: Error when checking model target: expected block5_maxpool to have 4 > dimensions, but got array with shape (16, 1)

因此，基本模型中的最后一个池化层似乎出了点问题.也许我在尝试将基本模型与顶级模型连接时做错了.

Therefore, it seems that something is wrong with the last pooling layer in base model. Or probably I've done something wrong trying to connect base model with the top one.

有人有类似的问题吗?或者，也许有更好的方法来构建这种级联"模型?我正在将keras==2.0.0与theano后端一起使用.

Does anybody have similar issue? Or maybe there is a better way to build such "concatenated" models? I am using keras==2.0.0 with theano backend.

注意:我正在使用gist和applications.VGG16实用程序中的示例，但是在尝试连接模型时遇到问题，我对keras功能性API不太熟悉.因此，我在这里提供的解决方案是最成功"的解决方案，即仅在安装阶段失败.

Note: I was using examples from gist and applications.VGG16 utility, but has issues trying to concatenate models, I am not too familiar with keras functional API. So this solution I provide here is the most "successful" one, i.e. it fails only on fitting stage.

---

更新#1

好的，这是关于我要做什么的简短说明.首先，我将从VGG16生成瓶颈功能，如下所示:

---

Update #1

Ok, here is a small explanation about what I am trying to do. First of all, I am generating bottleneck features from VGG16 as follows:

def save_bottleneck_features():
    datagen = ImageDataGenerator(rescale=1./255)
    model = applications.VGG16(include_top=False, weights='imagenet')

    generator = datagen.flow_from_directory(
        TRAIN_DIR,
        target_size=(150, 150),
        batch_size=batch_size,
        class_mode=None,
        shuffle=False)    
    print("Predicting train samples..")
    bottleneck_features_train = model.predict_generator(generator, nb_train_samples)
    np.save(open('bottleneck_features_train.npy', 'w'), bottleneck_features_train)

    generator = datagen.flow_from_directory(
        VALID_DIR,
        target_size=(150, 150),
        batch_size=batch_size,
        class_mode=None,
        shuffle=False)
    print("Predicting valid samples..")
    bottleneck_features_valid = model.predict_generator(generator, nb_valid_samples)
    np.save(open('bottleneck_features_valid.npy', 'w'), bottleneck_features_valid)

然后，我创建一个顶级模型，并按照以下功能对其进行训练:

Then, I create a top model and train it on these features as follows:

def train_top_model():
    train_data = np.load(open('bottleneck_features_train.npy'))
    train_labels = np.array([0]*(nb_train_samples / 2) + 
                            [1]*(nb_train_samples / 2))
    valid_data = np.load(open('bottleneck_features_valid.npy'))
    valid_labels = np.array([0]*(nb_valid_samples / 2) +
                            [1]*(nb_valid_samples / 2))
    model = Sequential()
    model.add(Flatten(input_shape=train_data.shape[1:]))  
    model.add(Dense(256, activation='relu'))
    model.add(Dropout(0.5))
    model.add(Dense(1, activation='sigmoid'))
    model.compile(optimizer='rmsprop', loss='binary_crossentropy', metrics=['accuracy'])
    model.fit(train_data, train_labels,
              nb_epoch=nb_epoch,
              batch_size=batch_size,
              validation_data=(valid_data, valid_labels),
              verbose=1)
    model.save_weights('top_model.h5')   

因此，基本上，有两个训练有素的模型:具有ImageNet权重的base_model和具有瓶颈特性的权重的top_model.而且我想知道如何将它们串联起来?有可能还是我做错了什么?因为如我所见，@ thomas-pinetz的响应假设顶级模型没有单独训练，而是立即附加到模型.不知道我是否清楚，这是博客的引文:

So basically, there are two trained models, base_model with ImageNet weights and top_model with weights generated from bottleneck features. And I wonder how to concatenate them? Is it possible or I am doing something wrong? Because as I can see, the response from @thomas-pinetz supposes that the top model is not trained separately and right away appended to the model. Not sure if I am clear, here is a quote from the blog:

为了执行微调，所有层都应从经过适当训练的权重开始:例如，您不应在预先训练的卷积基础上拍打随机初始化的完全连接的网络.这是因为由随机初始化的权重触发的大梯度更新将破坏卷积基础中的学习权重.在我们的案例中，这就是为什么我们首先训练顶级分类器，然后才开始对它进行微调卷积权重的原因.

In order to perform fine-tuning, all layers should start with properly trained weights: for instance you should not slap a randomly initialized fully-connected network on top of a pre-trained convolutional base. This is because the large gradient updates triggered by the randomly initialized weights would wreck the learned weights in the convolutional base. In our case this is why we first train the top-level classifier, and only then start fine-tuning convolutional weights alongside it.

推荐答案

我认为vgg网络描述的权重不适合您的模型，因此产生错误.无论如何，有一种更好的方法可以使用网络本身来完成此操作，如( https://keras.io /applications/#vgg16 ).

I think that the weights described by the vgg net do not fit your model and the error stems from this. Anyways there is a way better way to do this using the network itself as described in (https://keras.io/applications/#vgg16).

您可以使用:

base_model = keras.applications.vgg16.VGG16(include_top=False, weights='imagenet', input_tensor=None, input_shape=None)

实例化经过预训练的vgg网络.然后，您可以冻结图层并使用模型类实例化自己的模型，如下所示:

to instantiate a vgg net that is pre-trained. Then you can freeze the layers and use the model class to instantiate your own model like this:

x = base_model.output
x = Flatten()(x)
x = Dense(your_classes, activation='softmax')(x) #minor edit
new_model = Model(input=base_model.input, output=x)

要结合最底层和最顶层的网络，可以使用以下代码片段.使用了以下功能(输入层( https://keras.io/getting- starts/functional-api-guide/)/load_model( https://keras.io/getting-started/faq/#how-can-i-save-a-keras-model )和keras的功能API):

To combine the bottom and the top network you can use the following code snippet. The following functions are used (Input Layer (https://keras.io/getting-started/functional-api-guide/) / load_model (https://keras.io/getting-started/faq/#how-can-i-save-a-keras-model) and the functional API of keras):

final_input = Input(shape=(3, 224, 224))
base_model = vgg...
top_model = load_model(weights_file)

x = base_model(final_input)
result = top_model(x)
final_model = Model(input=final_input, output=result)

这篇关于Keras VGG16微调的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！