混淆矩阵给出不好的结果，但验证准确度约为95％ [英] Confusion Matrix giving Bad Results but Validation Accuracy ~95%

问题描述

这是我的代码，我在培训中大约有5000张图像，在测试数据中大约有532张图像.我的Val_accuracy显示95％，但是当我创建混淆矩阵和分类报告时，在532张图像中，它预测314正确(TP)时，在验证/测试集上给出的结果非常差.我认为问题出在设置batch_size和其他超参数.请帮助，这是给我的研究论文.请帮忙，我卡得很厉害！

This is my Code, I have around 5000 images in Training and roughly 532 in test data. My Val_accuracy shows 95% but when i create Confusion matrix and classification report, it gives very poor results on validation/test set, out of 532 images it predicts 314 correct (TP). I think the problem lies in setting batch_size and other hyperparameters. Please HELP, This is for my Research Paper. Please help, I'M stuck badly!

import os
import numpy as np
import matplotlib.pyplot as plt
import keras
from keras.applications import xception
from keras.layers import *
from keras.models import *
from keras.preprocessing import image

model = xception.Xception(weights='imagenet', include_top=False, input_shape=(224, 224, 3))
for layers in model.layers:
    layers.trainable=False
    
flat1 = Flatten()(model.layers[-1].output)
class1 = Dense(256, activation='relu')(flat1)
output = Dense(1, activation='sigmoid')(class1)

model = Model(inputs = model.inputs, outputs = output)


model.compile(loss = 'binary_crossentropy', optimizer='adam', metrics=['accuracy'])


train_datagen = image.ImageDataGenerator(
    rescale = 1./255,
    shear_range = 0.2,
    zoom_range = 0.2,
    horizontal_flip = True,
    )

test_datagen = image.ImageDataGenerator(rescale = 1./255)

train_generator = train_datagen.flow_from_directory(
    '/Users/xd_anshul/Desktop/Research/Major/CovidDataset/Train',
    target_size = (224,224),
    batch_size = 10,
    class_mode='binary')

validation_generator = test_datagen.flow_from_directory(
    '/Users/xd_anshul/Desktop/Research/Major/CovidDataset/Test',
    target_size = (224,224),
    batch_size = 10,
    class_mode='binary')



hist = model.fit(
    train_generator,
    steps_per_epoch=9,
    epochs=5,
    validation_data=validation_generator,
    validation_steps=2)






from sklearn.metrics import classification_report, confusion_matrix
Y_pred = model.predict(validation_generator)
y_pred = [1 * (x[0]>=0.5) for x in Y_pred]
print('Confusion Matrix')
print(confusion_matrix(validation_generator.classes, y_pred))
print('Classification Report')
target_names = ['Covid', 'Normal']
print(classification_report(validation_generator.classes, y_pred, 
target_names=target_names))

OUTPUT:

Epoch 1/5
9/9 [==============================] - 21s 2s/step - loss: 0.2481 - accuracy: 0.9377 - val_loss: 4.1552 - val_accuracy: 0.9500
Epoch 2/5
9/9 [==============================] - 16s 2s/step - loss: 1.9680 - accuracy: 0.9767 - val_loss: 15.5336 - val_accuracy: 0.8500
Epoch 3/5
9/9 [==============================] - 16s 2s/step - loss: 0.2898 - accuracy: 0.9867 - val_loss: 0.0000e+00 - val_accuracy: 1.0000
Epoch 4/5
9/9 [==============================] - 16s 2s/step - loss: 1.4597 - accuracy: 0.9640 - val_loss: 2.3671 - val_accuracy: 0.9500
Epoch 5/5
9/9 [==============================] - 16s 2s/step - loss: 3.3822 - accuracy: 0.9365 - val_loss: 3.5101e-22 - val_accuracy: 1.0000


  
  Confusion Matrix
[[314  96]
 [ 93  29]]
Classification Report
              precision    recall  f1-score   support

       Covid       0.77      0.77      0.77       410
      Normal       0.23      0.24      0.23       122

    accuracy                           0.64       532
   macro avg       0.50      0.50      0.50       532
weighted avg       0.65      0.64      0.65       532

推荐答案

假设您的预测数组类似于:

Let's say your predictions array is something like:

preds_sigmoid = np.array([[0.8451], [0.454], [0.5111]])

包含这些值的

作为S型，将它们压缩在[0,1]的范围内.当您像以前一样应用 argmax 时，由于 argmax 返回指定轴上的最大索引，因此每次都会获得索引0.

containing these values as sigmoid squeeze them in a range of [0,1]. When you apply argmax as you did, you will get index 0 everytime because argmax returns the maximum index at specified axis.

pred = np.argmax(preds_sigmoid , axis = 1) # pred is full of zeros.

您应该对预测进行评估，例如它是否大于某个阈值，比如说0.5，它属于第二类.您可以为此使用列表理解:

You should evaluate the predictions like if it is bigger than some threshold, let's say 0.5, it belongs to second class. You can use list comprehension for this:

pred = [1 * (x[0]>=0.5) for x in preds_sigmoid]

因此，预测将得到正确处理.

Therefore predictions will be handled properly.

这篇关于混淆矩阵给出不好的结果，但验证准确度约为95％的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！