绘制K折交叉验证的ROC曲线 [英] Plotting the ROC curve of K-fold Cross Validation

问题描述

我是机器学习的新手。我正在使用不平衡的数据集。在将数据集分为测试集和训练集之前，我已应用SMOTE算法来平衡数据集，然后再应用ML模型。我想应用交叉验证并绘制每个折叠的ROC曲线，以显示每个折叠的AUC，并在图中显示AUC的平均值。我将重新采样的训练集变量命名为X_train_res和y_train_res，下面是代码：

I am new to Machine Learning. I am working with an imbalanced dataset. I have applied SMOTE Algorithm to balance the dataset after splitting the dataset into test and training set before applying ML models. I want to apply cross-validation and plot the ROC curves of each folds showing the AUC of each fold and also display the mean of the AUCs in the plot. I named the resampled training set variables as X_train_res and y_train_res and following is the code:

cv = StratifiedKFold(n_splits=10)
classifier = SVC(kernel='sigmoid',probability=True,random_state=0)

tprs = []
aucs = []
mean_fpr = np.linspace(0, 1, 100)
plt.figure(figsize=(10,10))
i = 0
for train, test in cv.split(X_train_res, y_train_res):
    probas_ = classifier.fit(X_train_res[train], y_train_res[train]).predict_proba(X_train_res[test])
    # Compute ROC curve and area the curve
    fpr, tpr, thresholds = roc_curve(y_train_res[test], probas_[:, 1])
    tprs.append(interp(mean_fpr, fpr, tpr))
    tprs[-1][0] = 0.0
    roc_auc = auc(fpr, tpr)
    aucs.append(roc_auc)
    plt.plot(fpr, tpr, lw=1, alpha=0.3,
             label='ROC fold %d (AUC = %0.2f)' % (i, roc_auc))

    i += 1
plt.plot([0, 1], [0, 1], linestyle='--', lw=2, color='r',
         label='Chance', alpha=.8)

mean_tpr = np.mean(tprs, axis=0)
mean_tpr[-1] = 1.0
mean_auc = auc(mean_fpr, mean_tpr)
std_auc = np.std(aucs)
plt.plot(mean_fpr, mean_tpr, color='b',
         label=r'Mean ROC (AUC = %0.2f $\pm$ %0.2f)' % (mean_auc, std_auc),
         lw=2, alpha=.8)

std_tpr = np.std(tprs, axis=0)
tprs_upper = np.minimum(mean_tpr + std_tpr, 1)
tprs_lower = np.maximum(mean_tpr - std_tpr, 0)
plt.fill_between(mean_fpr, tprs_lower, tprs_upper, color='grey', alpha=.2,
                 label=r'$\pm$ 1 std. dev.')

plt.xlim([-0.01, 1.01])
plt.ylim([-0.01, 1.01])
plt.xlabel('False Positive Rate',fontsize=18)
plt.ylabel('True Positive Rate',fontsize=18)
plt.title('Cross-Validation ROC of SVM',fontsize=18)
plt.legend(loc="lower right", prop={'size': 15})
plt.show()

以下是输出：

请告诉我代码对于交叉验证绘制ROC曲线是否正确。

Please tell me whether the code is correct for plotting ROC curve for the cross-validation or not.

推荐答案

问题是我不清楚交叉验证。在for循环范围内，我通过了X和y变量的训练集。交叉验证是否可以这样工作？

The problem is that I do not clearly understand cross-validation. In the for loop range, I have passed the training sets of X and y variables. Does cross-validation work like this?

确实如此，您的过程看起来正确。

Indeed it does, and your procedure looks correct.

更详细地，对于每个 n_splits = 10 ：

• 您创建火车和 test 折叠



• 您可以使用火车折叠来拟合模型：

• you create train and test folds

• you fit the model using the train fold:

classifier.fit(X_train_res[train], y_train_res[train])

然后使用 test 折叠来预测概率：

 predict_proba(X_train_res[test])

因此，由于您有 n_splits = 10 ，因此您将获得10条ROC曲线以及相应的完全符合预期的AUC值（及其平均值）。

So, since you have n_splits=10, you get 10 ROC curves and respective AUC values (and their average), exactly as expected.

这篇关于绘制K折交叉验证的ROC曲线的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！