验证准确性很高，但预测不正确 [英] Very good validation accuracy but bad predictions

问题描述

我正在建立一个keras模型来对猫和狗进行分类.我使用了具有瓶颈功能的转移学习和vgg模型的微调.现在，我获得了非常不错的验证准确性，例如97％，但是当我进行预测时，关于分类报告和混淆矩阵的结果却非常糟糕.可能是什么问题?

I'm building a keras model to classify cats and dogs. I used transfer learning with bottleneck features and fine tuning with vgg model. Now I get very good validation accuracy like 97% but when I get to predict I get very bad results regarding the classification report and confusion matrix. What could be the problem?

这是微调的代码和我得到的结果

Here is the code of fine tuning and the results I get

base_model = applications.VGG16(weights='imagenet', include_top=False, input_shape=(150,150,3))
print('Model loaded.')

# build a classifier model to put on top of the convolutional model
top_model = Sequential()
top_model.add(Flatten(input_shape=base_model.output_shape[1:]))
top_model.add(Dense(256, activation='relu'))
top_model.add(Dropout(0.5))
top_model.add(Dense(2, activation='sigmoid'))

# note that it is necessary to start with a fully-trained
# classifier, including the top classifier,
# in order to successfully do fine-tuning
top_model.load_weights(top_model_weights_path)

# add the model on top of the convolutional base
# model.add(top_model)
model = Model(inputs=base_model.input, outputs=top_model(base_model.output))

# set the first 25 layers (up to the last conv block)
# to non-trainable (weights will not be updated)
for layer in model.layers[:15]:
    layer.trainable = False

# compile the model with a SGD/momentum optimizer
# and a very slow learning rate.
model.compile(loss='binary_crossentropy',
              optimizer=optimizers.SGD(lr=1e-4, momentum=0.9),
              metrics=['accuracy'])

# prepare data augmentation configuration
train_datagen = ImageDataGenerator(
    rescale=1. / 255,
    shear_range=0.2,
    zoom_range=0.2,
    horizontal_flip=True)

test_datagen = ImageDataGenerator(rescale=1. / 255)

train_generator = train_datagen.flow_from_directory(
    train_data_dir,
    target_size=(img_height, img_width),
    batch_size=batch_size,
    class_mode='categorical')

validation_generator = test_datagen.flow_from_directory(
    validation_data_dir,
    target_size=(img_height, img_width),
    batch_size=batch_size,
    class_mode='categorical')

model.summary()

# fine-tune the model
model.fit_generator(
    train_generator,
    steps_per_epoch=nb_train_samples // batch_size,
    epochs=epochs,
    validation_data=validation_generator,
    validation_steps=nb_validation_samples // batch_size,
    verbose=2)
scores=model.evaluate_generator(generator=validation_generator,
steps=nb_validation_samples // batch_size)
print("Accuracy = ", scores[1])

Y_pred = model.predict_generator(validation_generator, nb_validation_samples // batch_size)

y_pred = np.argmax(Y_pred, axis=1)

print('Confusion Matrix')

print(confusion_matrix(validation_generator.classes, y_pred))

print('Classification Report')

target_names = ['Cats', 'Dogs']

print(classification_report(validation_generator.classes, y_pred, target_names=target_names))
model.save("model_tuned.h5")

准确度= 0.974375

Accuracy = 0.974375

混淆矩阵[[186 214][199 201]]

Confusion Matrix [[186 214] [199 201]]

分类报告

          precision    recall  f1-score   support

    Cats       0.48      0.47      0.47       400
    Dogs       0.48      0.50      0.49       400

微型平均0.48 0.48 0.48 800宏观平均0.48 0.48 0.48 800加权平均0.48 0.48 0.48 800

micro avg 0.48 0.48 0.48 800 macro avg 0.48 0.48 0.48 800 weighted avg 0.48 0.48 0.48 800

推荐答案

我认为问题在于您应该在验证生成器中添加shuffle = False

I think the problem is that you should add shuffle = False in your validation generator

validation_generator = test_datagen.flow_from_directory(
validation_data_dir,
target_size=(img_height, img_width),
batch_size=batch_size,
class_mode='categorical',
shuffle=False)

问题在于默认行为是将图像随机播放，因此标签顺序为

The problem is that the default behaviour is to shuffle the images so the label order of

validation_generator.classes

与生成器不匹配

这篇关于验证准确性很高，但预测不正确的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！