在CNN评估期间如何存储错误的预测 [英] How to store wrong predictions during evaluation on the CNN

问题描述

在评估期间，我想存储被错误地预测会进行更多处理的唯一ID.这是一个多类预测问题

During evaluation, I want to store unique ids that are wrongly predicted to do some more processing. It is a multiclass prediction problem

以下是评估过程中的代码:

Here is the code during the evaluation:

 outputs = model(imgs)
 loss = criterion(outputs, targets)  # Prediction error

 val_loss += loss.item()
 predicted = torch.argmax(outputs, dim=1)
 t_predicted +=predicted.cpu().tolist()

 total += targets.size(0)
 good_answers = (predicted == targets)
 correct += good_answers.sum().item()

知道 ids 是图像ID的列表，当我尝试获取错误的ID时:

Knowing that ids is a list of the ids of the images, When I try to get the ids that are wrong:

wrong_ids += ids[~(good_answers.to('cpu'))]

我收到此错误:

add(): argument 'other' (position 1) must be Tensor

推荐答案

该问题包含一个 tensorflow 标记，因此我正在准备一个答案.完成写操作后，我发现该标签已删除.但是，我相信我的回答可以深入了解这个一般性问题，即他们使用的是 tf 还是 pytorch .

The question contained a tensorflow tag, so I was preparing an answer. After completing my write up, I've found that this tag is removed. However, I believe my answer can give insight into this general question of whether they're using tf or pytorch.

(x_train, y_train), (x_test, y_test) = tf.keras.datasets.cifar10.load_data()

# train set / data 
x_train = x_train.astype('float32') / 255

# validation set / data 
x_test = x_test.astype('float32') / 255

# train set / target 
y_train = tf.keras.utils.to_categorical(y_train, num_classes=10)
# validation set / target 
y_test = tf.keras.utils.to_categorical(y_test, num_classes=10)

print(x_train.shape, y_train.shape)
print(x_test.shape, y_test.shape)  

火车

import tensorflow as tf 

# declare input shape 
input = tf.keras.Input(shape=(32,32,3))
# Block 1
x = tf.keras.layers.Conv2D(32, 3, strides=2, activation="relu")(input)
x = tf.keras.layers.MaxPooling2D(3)(x)
# Now that we apply global max pooling.
gap = tf.keras.layers.GlobalMaxPooling2D()(x)
# Finally, we add a classification layer.
output = tf.keras.layers.Dense(10, activation='softmax')(gap)
# bind all
func_model = tf.keras.Model(input, output)

print('\nFunctional API')
func_model.compile(
          metrics=['accuracy'],
          loss      = 'categorical_crossentropy',
          optimizer = tf.keras.optimizers.Adam()
          )

func_model.fit(x_train, y_train)

错误预测

# Predict the values from the validation dataset
y_pred = func_model.predict(x_test)
# Convert predictions classes to one hot vectors 
y_pred_classes = np.argmax(y_pred, axis = 1) 
y_test = np.argmax(y_test, axis=1)

# Errors are difference between predicted labels and true labels
errors = (y_pred_classes - y_test != 0)

y_pred_classes_errors = y_pred_classes[errors]
y_pred_errors = y_pred[errors]
y_true_errors = y_test[errors]
x_test_errors = x_test[errors]

# Probabilities of the wrong predicted numbers
y_pred_errors_prob = np.max(y_pred_errors, axis = 1)

# Predicted probabilities of the true values in the error set
true_prob_errors = np.diagonal(np.take(y_pred_errors, y_true_errors, axis=1))

# Difference between the probability of the predicted label and the true label
delta_pred_true_errors = y_pred_errors_prob - true_prob_errors

# Sorted list of the delta prob errors
sorted_dela_errors = np.argsort(delta_pred_true_errors)

# Top 6 errors 
most_important_errors = sorted_dela_errors[-6:]

显示

import matplotlib.pyplot as plt

def display_errors(errors_index,img_errors,pred_errors, obs_errors):
    """ This function shows 6 images with their predicted and real labels"""
    n = 0
    nrows = 2
    ncols = 3
    fig, ax = plt.subplots(nrows,ncols,sharex=True,sharey=True)
    for row in range(nrows):
        for col in range(ncols):
            error = errors_index[n]
            ax[row,col].imshow((img_errors[error]).reshape((32,32,3)))
            ax[row,col].set_title("Predicted label :{}\nTrue label :{}".format(pred_errors[error],obs_errors[error]))
            n += 1

# Show the top 6 errors
display_errors(most_important_errors, x_test_errors, y_pred_classes_errors, y_true_errors)

这篇关于在CNN评估期间如何存储错误的预测的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！