混淆矩阵给出了糟糕的结果，但验证准确度约为 95% [英] Confusion Matrix giving Bad Results but Validation Accuracy ~95%

问题描述

这是我的代码，我有大约 5000 张训练图像和大约 532 张测试数据.我的 Val_accuracy 显示为 95%，但是当我创建混淆矩阵和分类报告时，它在验证/测试集上给出了非常差的结果，在 532 张图像中它预测 314 个正确(TP).我认为问题在于设置 batch_size 和其他超参数.请帮助，这是我的研究论文.请帮忙，我被卡住了！

This is my Code, I have around 5000 images in Training and roughly 532 in test data. My Val_accuracy shows 95% but when i create Confusion matrix and classification report, it gives very poor results on validation/test set, out of 532 images it predicts 314 correct (TP). I think the problem lies in setting batch_size and other hyperparameters. Please HELP, This is for my Research Paper. Please help, I'M stuck badly!

import os
import numpy as np
import matplotlib.pyplot as plt
import keras
from keras.applications import xception
from keras.layers import *
from keras.models import *
from keras.preprocessing import image

model = xception.Xception(weights='imagenet', include_top=False, input_shape=(224, 224, 3))
for layers in model.layers:
    layers.trainable=False
    
flat1 = Flatten()(model.layers[-1].output)
class1 = Dense(256, activation='relu')(flat1)
output = Dense(1, activation='sigmoid')(class1)

model = Model(inputs = model.inputs, outputs = output)


model.compile(loss = 'binary_crossentropy', optimizer='adam', metrics=['accuracy'])


train_datagen = image.ImageDataGenerator(
    rescale = 1./255,
    shear_range = 0.2,
    zoom_range = 0.2,
    horizontal_flip = True,
    )

test_datagen = image.ImageDataGenerator(rescale = 1./255)

train_generator = train_datagen.flow_from_directory(
    '/Users/xd_anshul/Desktop/Research/Major/CovidDataset/Train',
    target_size = (224,224),
    batch_size = 10,
    class_mode='binary')

validation_generator = test_datagen.flow_from_directory(
    '/Users/xd_anshul/Desktop/Research/Major/CovidDataset/Test',
    target_size = (224,224),
    batch_size = 10,
    class_mode='binary')



hist = model.fit(
    train_generator,
    steps_per_epoch=9,
    epochs=5,
    validation_data=validation_generator,
    validation_steps=2)






from sklearn.metrics import classification_report, confusion_matrix
Y_pred = model.predict(validation_generator)
y_pred = [1 * (x[0]>=0.5) for x in Y_pred]
print('Confusion Matrix')
print(confusion_matrix(validation_generator.classes, y_pred))
print('Classification Report')
target_names = ['Covid', 'Normal']
print(classification_report(validation_generator.classes, y_pred, 
target_names=target_names))

OUTPUT:

Epoch 1/5
9/9 [==============================] - 21s 2s/step - loss: 0.2481 - accuracy: 0.9377 - val_loss: 4.1552 - val_accuracy: 0.9500
Epoch 2/5
9/9 [==============================] - 16s 2s/step - loss: 1.9680 - accuracy: 0.9767 - val_loss: 15.5336 - val_accuracy: 0.8500
Epoch 3/5
9/9 [==============================] - 16s 2s/step - loss: 0.2898 - accuracy: 0.9867 - val_loss: 0.0000e+00 - val_accuracy: 1.0000
Epoch 4/5
9/9 [==============================] - 16s 2s/step - loss: 1.4597 - accuracy: 0.9640 - val_loss: 2.3671 - val_accuracy: 0.9500
Epoch 5/5
9/9 [==============================] - 16s 2s/step - loss: 3.3822 - accuracy: 0.9365 - val_loss: 3.5101e-22 - val_accuracy: 1.0000


  
  Confusion Matrix
[[314  96]
 [ 93  29]]
Classification Report
              precision    recall  f1-score   support

       Covid       0.77      0.77      0.77       410
      Normal       0.23      0.24      0.23       122

    accuracy                           0.64       532
   macro avg       0.50      0.50      0.50       532
weighted avg       0.65      0.64      0.65       532

推荐答案

假设你的预测数组类似于:

Let's say your predictions array is something like:

preds_sigmoid = np.array([[0.8451], [0.454], [0.5111]])

包含这些值作为 sigmoid 将它们压缩在 [0,1] 的范围内.当您像这样应用 argmax 时，每次都会得到索引 0，因为 argmax 返回指定轴上的最大索引.

containing these values as sigmoid squeeze them in a range of [0,1]. When you apply argmax as you did, you will get index 0 everytime because argmax returns the maximum index at specified axis.

pred = np.argmax(preds_sigmoid , axis = 1) # pred is full of zeros.

您应该评估预测，例如它是否大于某个阈值，例如 0.5，它属于第二类.您可以为此使用列表理解:

You should evaluate the predictions like if it is bigger than some threshold, let's say 0.5, it belongs to second class. You can use list comprehension for this:

pred = [1 * (x[0]>=0.5) for x in preds_sigmoid]

因此将正确处理预测.

这篇关于混淆矩阵给出了糟糕的结果，但验证准确度约为 95%的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！