SKlearn中具有嵌套交叉验证的分类报告(平均值/单个值) [英] Classification report with Nested Cross Validation in SKlearn (Average/Individual values)
问题描述
是否可以通过一些变通办法从cross_val_score获取分类报告?我正在使用嵌套的交叉验证,在这里可以获得模型的各种评分,但是,我希望看到外部循环的分类报告.有什么建议吗?
Is it possible to get classification report from cross_val_score through some workaround? I'm using nested cross-validation and I can get various scores here for a model, however, I would like to see the classification report of the outer loop. Any recommendations?
# Choose cross-validation techniques for the inner and outer loops,
# independently of the dataset.
# E.g "LabelKFold", "LeaveOneOut", "LeaveOneLabelOut", etc.
inner_cv = KFold(n_splits=4, shuffle=True, random_state=i)
outer_cv = KFold(n_splits=4, shuffle=True, random_state=i)
# Non_nested parameter search and scoring
clf = GridSearchCV(estimator=svr, param_grid=p_grid, cv=inner_cv)
# Nested CV with parameter optimization
nested_score = cross_val_score(clf, X=X_iris, y=y_iris, cv=outer_cv)
我想在这里看到分数值旁边的分类报告. http://scikit-learn.org/stable/modules/genic /sklearn.metrics.classification_report.html
I would like to see a classification report here along side the score values. http://scikit-learn.org/stable/modules/generated/sklearn.metrics.classification_report.html
推荐答案
我们可以定义自己的评分函数,如下所示:
We can define our own scoring function as below:
from sklearn.metrics import classification_report, accuracy_score, make_scorer
def classification_report_with_accuracy_score(y_true, y_pred):
print classification_report(y_true, y_pred) # print classification report
return accuracy_score(y_true, y_pred) # return accuracy score
现在,只需使用make_scorer
使用新的评分功能调用cross_val_score
:
Now, just call cross_val_score
with our new scoring function, using make_scorer
:
# Nested CV with parameter optimization
nested_score = cross_val_score(clf, X=X_iris, y=y_iris, cv=outer_cv, \
scoring=make_scorer(classification_report_with_accuracy_score))
print nested_score
它将分类报告作为文本打印,同时将nested_score
作为数字返回.
It will print the classification report as text at the same time return the nested_score
as a number.
http://scikit-learn.org/stable/auto_examples/model_selection/使用此新评分功能运行时,plot_nested_cross_validation_iris.html 示例,输出的最后几行如下:
http://scikit-learn.org/stable/auto_examples/model_selection/plot_nested_cross_validation_iris.html example when run with this new scoring function, the last few lines of the output will be as follows:
# precision recall f1-score support
#0 1.00 1.00 1.00 14
#1 1.00 1.00 1.00 14
#2 1.00 1.00 1.00 9
#avg / total 1.00 1.00 1.00 37
#[ 0.94736842 1. 0.97297297 1. ]
#Average difference of 0.007742 with std. dev. of 0.007688.
这篇关于SKlearn中具有嵌套交叉验证的分类报告(平均值/单个值)的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!