Keras:精度在开始调整时会下降 [英] Keras: Accuracy Drops While Finetuning Inception

问题描述

我在用Keras调整Inception模型时遇到麻烦.

I am having trouble fine tuning an Inception model with Keras.

我设法使用教程和文档生成了一个完全连接的顶层模型，该模型使用Inception的瓶颈功能将数据集分类为适当的类别，准确率超过99％.

I have managed to use tutorials and documentation to generate a model of fully connected top layers that classifies my dataset into their proper categories with an accuracy over 99% using bottleneck features from Inception.

import numpy as np
from keras.preprocessing.image import ImageDataGenerator
from keras.models import Sequential
from keras.layers import Dropout, Flatten, Dense
from keras import applications


# dimensions of our images.
img_width, img_height = 150, 150

#paths for saving weights and finding datasets
top_model_weights_path = 'Inception_fc_model_v0.h5'
train_data_dir = '../data/train2'
validation_data_dir = '../data/train2' 

#training related parameters?
inclusive_images = 1424
nb_train_samples = 1424
nb_validation_samples = 1424
epochs = 50
batch_size = 16


def save_bottlebeck_features():
    datagen = ImageDataGenerator(rescale=1. / 255)

    # build bottleneck features
    model = applications.inception_v3.InceptionV3(include_top=False, weights='imagenet', input_shape=(img_width,img_height,3))

    generator = datagen.flow_from_directory(
        train_data_dir,
        target_size=(img_width, img_height),
        batch_size=batch_size,
        class_mode='categorical',
        shuffle=False)

    bottleneck_features_train = model.predict_generator(
        generator, nb_train_samples // batch_size)

    np.save('bottleneck_features_train', bottleneck_features_train)

    generator = datagen.flow_from_directory(
        validation_data_dir,
        target_size=(img_width, img_height),
        batch_size=batch_size,
        class_mode='categorical',
        shuffle=False)

    bottleneck_features_validation = model.predict_generator(
        generator, nb_validation_samples // batch_size)

    np.save('bottleneck_features_validation', bottleneck_features_validation)

def train_top_model():
    train_data = np.load('bottleneck_features_train.npy')
    train_labels = np.array(range(inclusive_images))

    validation_data = np.load('bottleneck_features_validation.npy')
    validation_labels = np.array(range(inclusive_images))

    print('base size ', train_data.shape[1:])

    model = Sequential()
    model.add(Flatten(input_shape=train_data.shape[1:]))
    model.add(Dense(1000, activation='relu'))
    model.add(Dense(inclusive_images, activation='softmax'))
    model.compile(loss='sparse_categorical_crossentropy',
             optimizer='Adam',
             metrics=['accuracy'])

    proceed = True

    #model.load_weights(top_model_weights_path)

    while proceed:
        history = model.fit(train_data, train_labels,
              epochs=epochs,
              batch_size=batch_size)#,
              #validation_data=(validation_data, validation_labels), verbose=1)
        if history.history['acc'][-1] > .99:
            proceed = False

    model.save_weights(top_model_weights_path)


save_bottlebeck_features()
train_top_model()

第50/50集 1424/1424 [==============================]-17s 12ms/step-损失:0.0398-acc:0.9909

Epoch 50/50 1424/1424 [==============================] - 17s 12ms/step - loss: 0.0398 - acc: 0.9909

我还能够在开始时就将此模型堆叠起来以创建我的完整模型，并使用该完整模型成功地对我的训练集进行分类.

I have also been able to stack this model on top of inception to create my full model and use that full model to successfully classify my training set.

from keras import Model
from keras import optimizers
from keras.callbacks import EarlyStopping

img_width, img_height = 150, 150

top_model_weights_path = 'Inception_fc_model_v0.h5'
train_data_dir = '../data/train2'
validation_data_dir = '../data/train2' 

#how many inclusive examples do we have?
inclusive_images = 1424
nb_train_samples = 1424
nb_validation_samples = 1424
epochs = 50
batch_size = 16

# build the complete network for evaluation
base_model = applications.inception_v3.InceptionV3(weights='imagenet', include_top=False, input_shape=(img_width,img_height,3))

top_model = Sequential()
top_model.add(Flatten(input_shape=base_model.output_shape[1:]))
top_model.add(Dense(1000, activation='relu'))
top_model.add(Dense(inclusive_images, activation='softmax'))

top_model.load_weights(top_model_weights_path)

#combine base and top model
fullModel = Model(input= base_model.input, output= top_model(base_model.output))

#predict with the full training dataset
results = fullModel.predict_generator(ImageDataGenerator(rescale=1. / 255).flow_from_directory(
        train_data_dir,
        target_size=(img_width, img_height),
        batch_size=batch_size,
        class_mode='categorical',
        shuffle=False))

在此完整模型上处理的结果的检查与生成的完全连接模型的瓶颈的准确性相匹配.

inspection of the results from processing on this full model match the accuracy of the bottleneck generated fully connected model.

import matplotlib.pyplot as plt
import operator

#retrieve what the softmax based class assignments would be from results
resultMaxClassIDs = [ max(enumerate(result), key=operator.itemgetter(1))[0] for result in results]

#resultMaxClassIDs should be equal to range(inclusive_images) so we subtract the two and plot the log of the absolute value 
#looking for spikes that indicate the values aren't equal 
plt.plot([np.log(np.abs(x)+10) for x in (np.array(resultMaxClassIDs) - np.array(range(inclusive_images)))])

这是问题所在: 当我采用此完整模型并尝试对其进行训练时，即使验证仍保持在99％以上，准确性也会降至0.

Here is the problem: When I take this full model and attempt to train it, Accuracy drops to 0 even though validation remains above 99%.

model2 = fullModel

for layer in model2.layers[:-2]:
    layer.trainable = False

# compile the model with a SGD/momentum optimizer
# and a very slow learning rate.
#model.compile(loss='binary_crossentropy', optimizer=optimizers.SGD(lr=1e-4, momentum=0.9),  metrics=['accuracy'])

model2.compile(loss='categorical_crossentropy',
             optimizer=optimizers.SGD(lr=1e-4, momentum=0.9), 
             metrics=['accuracy'])

train_datagen = ImageDataGenerator(rescale=1. / 255)

test_datagen = ImageDataGenerator(rescale=1. / 255)

train_generator = train_datagen.flow_from_directory(
    train_data_dir,
    target_size=(img_height, img_width),
    batch_size=batch_size,
    class_mode='categorical')

validation_generator = test_datagen.flow_from_directory(
    validation_data_dir,
    target_size=(img_height, img_width),
    batch_size=batch_size,
    class_mode='categorical')

callback = [EarlyStopping(monitor='acc', min_delta=0, patience=3, verbose=0, mode='auto', baseline=None)]
# fine-tune the model
model2.fit_generator(
    #train_generator,
    validation_generator,
    steps_per_epoch=nb_train_samples//batch_size,
    validation_steps = nb_validation_samples//batch_size,
    epochs=epochs,
    validation_data=validation_generator)

历次1/50 89/89 [==============================]-388s 4s/step-损耗:13.5787-acc:0.0000e + 00-val_loss:0.0353-val_acc:0.9937

Epoch 1/50 89/89 [==============================] - 388s 4s/step - loss: 13.5787 - acc: 0.0000e+00 - val_loss: 0.0353 - val_acc: 0.9937

随着事情的进展，情况变得越来越糟

and it gets worse as things progress

第21/50集 89/89 [==============================]-372s 4s/step-损耗:7.3850-acc:0.0035-val_loss :0.5813-val_acc:0.8272

Epoch 21/50 89/89 [==============================] - 372s 4s/step - loss: 7.3850 - acc: 0.0035 - val_loss: 0.5813 - val_acc: 0.8272

我唯一能想到的是，在最后一趟火车上，训练标签被不正确地分配了，但是我之前已经使用VGG16用类似的代码成功地做到了这一点.

The only thing I could think of is that somehow the training labels are getting improperly assigned on this last train, but I've successfully done this with similar code using VGG16 before.

我已经搜索了所有代码，试图找到一个差异，以解释为什么模型在99％的时间内做出准确预测时会降低其训练精度，同时又在微调过程中保持验证精度，但是我无法弄清楚.任何帮助将不胜感激.

I have searched over the code trying to find a discrepancy to explain why a model making accurate predictions over 99% of the time drops its training accuracy while maintaining validation accuracy during fine tuning, but I can't figure it out. Any help would be appreciated.

有关代码和环境的信息:

Information about the code and environment:

有些奇怪的事，但事实就是这样:

Things that are going to stand out as weird, but are meant to be that way:

• 每个班级只有1张图片.该NN旨在进行分类 环境和定向条件为 受控.对于每个班级，它们只是一张可接受的图像 对应正确的环境和轮换情况.
• 测试集和验证集相同.这个NN只有 设计用于正在接受培训的课程.图像 它将处理该类示例的复本.是我的 意图使模型不适用于这些类

• There is only 1 image per class. This NN is intended to classify objects whose environmental and orientation conditions are controlled. Their is only one acceptable image for each class corresponding to the correct environmental and rotational situation.
• The test and validation set are the same. This NN is only ever designed to be used on the classes it is being trained on. The images it will process will be carbon copies of the class examples. It is my intent to overfit the model to these classes

我正在使用:

• Windows 10
• Anaconda客户端1.6.14下的Python 3.5.6
• Keras 2.2.2
• Tensorflow 1.10.0作为后端
• CUDA 9.0
• CuDNN 8.0

我已签出:

1. 精细调整模型中的Keras精度差异
2. VGG16 Keras微调:低精度
3. Keras:模型达到99％的准确度并损失0.01后，准确度下降
4. Keras Inception v3重新训练和微调错误
5. 如何查找哪个版本我的系统中已安装TensorFlow?

1. Keras accuracy discrepancy in fine-tuned model
2. VGG16 Keras fine tuning: low accuracy
3. Keras: model accuracy drops after reaching 99 percent accuracy and loss 0.01
4. Keras inception v3 retraining and finetuning error
5. How to find which version of TensorFlow is installed in my system?

但它们似乎无关.

推荐答案

注意:由于您的问题有点奇怪，如果没有训练有素的模型和数据集就很难调试，所以这个答案只是(最佳)猜测在考虑了许多可能会出错的事情之后.请提供您的反馈意见，如果该答案不起作用，我将删除它.

由于inception_V3包含BatchNormalization层，所以问题可能是由于将trainable参数设置为False时该层的行为(某种程度上是模棱两可或意外的)( 2 ， 3 ，

Since the inception_V3 contains BatchNormalization layers, maybe the problem is due to (somehow ambiguous or unexpected) behavior of this layer when you set trainable parameter to False (1, 2, 3, 4).

现在，让我们看看这是否是问题的根源: @fchollet 提出的建议，请在定义用于微调的模型时设置学习阶段:

Now, let's see if this is the root of the problem: as suggested by @fchollet, set the learning phase when defining the model for fine-tuning:

from keras import backend as K

K.set_learning_phase(0)

base_model = applications.inception_v3.InceptionV3(weights='imagenet', include_top=False, input_shape=(img_width,img_height,3))

for layer in base_model.layers:
    layer.trainable = False

K.set_learning_phase(1)

top_model = Sequential()
top_model.add(Flatten(input_shape=base_model.output_shape[1:]))
top_model.add(Dense(1000, activation='relu'))
top_model.add(Dense(inclusive_images, activation='softmax'))

top_model.load_weights(top_model_weights_path)

#combine base and top model
fullModel = Model(input= base_model.input, output= top_model(base_model.output))

fullModel.compile(loss='categorical_crossentropy',
             optimizer=optimizers.SGD(lr=1e-4, momentum=0.9), 
             metrics=['accuracy'])


#####################################################################
# Here, define the generators and then fit the model same as before #
#####################################################################

---

侧面说明:这不会对您造成任何问题，但请记住，当您使用top_model(base_model.output)时，整个顺序模型(即top_model)将存储为fullModel.您可以使用fullModel.summary()或print(fullModel.layers[-1])进行验证.因此，当您使用时:

---

Side Note: This is not causing any problem in your case, but keep in mind that when you use top_model(base_model.output) the whole Sequential model (i.e. top_model) is stored as one layer of fullModel. You can verify this by either using fullModel.summary() or print(fullModel.layers[-1]). Hence when you used:

for layer in model2.layers[:-2]:
    layer.trainable = False 

您实际上也没有冻结base_model的最后一层.但是，由于它是一个Concatenate层，因此没有可训练的参数，因此不会发生任何问题，并且它将按照您的预期运行.

you are actually not freezing the last layer of base_model as well. However, since it is a Concatenate layer, and therefore does not have trainable parameters, no problem occurs and it would behave as you intended.

这篇关于Keras:精度在开始调整时会下降的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！