如何访问Scikit了解嵌套的交叉验证分数 [英] How to access Scikit Learn nested cross-validation scores

问题描述

我正在使用python，并且我想在scikit learning中使用嵌套的交叉验证.我发现了一个很好的示例:

I'm using python and I would like to use nested cross-validation with scikit learn. I have found a very good example:

NUM_TRIALS = 30
non_nested_scores = np.zeros(NUM_TRIALS)
nested_scores = np.zeros(NUM_TRIALS)
# Choose cross-validation techniques for the inner and outer loops,
# independently of the dataset.
# E.g "LabelKFold", "LeaveOneOut", "LeaveOneLabelOut", etc.
inner_cv = KFold(n_splits=4, shuffle=True, random_state=i)
outer_cv = KFold(n_splits=4, shuffle=True, random_state=i)

# Non_nested parameter search and scoring
clf = GridSearchCV(estimator=svr, param_grid=p_grid, cv=inner_cv)
clf.fit(X_iris, y_iris)
non_nested_scores[i] = clf.best_score_

# Nested CV with parameter optimization
nested_score = cross_val_score(clf, X=X_iris, y=y_iris, cv=outer_cv)
nested_scores[i] = nested_score.mean()

如何才能访问嵌套交叉验证中的最佳参数集以及所有参数集(及其对应的分数)?

How can the best set of parameters as well as all set of parameters (with their corresponding score) from the nested cross-validation be accessed?

推荐答案

您无法从cross_val_score访问单个参数和最佳参数. cross_val_score在内部所做的是克隆提供的估计量，然后在各个估计量上给定的X，y对其调用fit和score方法.

You cannot access individual params and best params from cross_val_score. What cross_val_score does internally is clone the supplied estimator and then call fit and score methods on it with given X, y on individual estimators.

如果要在每次拆分时访问参数，则可以使用:

If you want to access the params at each split you can use:

#put below code inside your NUM_TRIALS for loop
cv_iter = 0
temp_nested_scores_train = np.zeros(4)
temp_nested_scores_test = np.zeros(4)
for train, test in outer_cv.split(X_iris):
    clf.fit(X_iris[train], y_iris[train])
    temp_nested_scores_train[cv_iter] = clf.best_score_
    temp_nested_scores_test[cv_iter] = clf.score(X_iris[test], y_iris[test])
    #You can access grid search's params here
nested_scores_train[i] = temp_nested_scores_train.mean()
nested_scores_test[i] = temp_nested_scores_test.mean()

这篇关于如何访问Scikit了解嵌套的交叉验证分数的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！