如何从k折交叉验证的每个折中学习? [英] how to learn from each fold in the k-fold cross validation?

问题描述

执行k折交叉验证时，对于每折，我们都有不同的验证集和稍有变化的学习集.假设您从第一折前进到第二折.我们从第一折中学到的东西如何在第二折迭代中继承?目前，似乎您只计算准确性，而学习的模型却被丢弃并且永远不会保留.

When performing k-fold cross-validation, for every fold, we have a different validation set and a slightly changed learning set. Say that you progress from the first fold to the second fold. How is what υοu learned from the first fold being inherited in the second fold iteration? Currently, it seems as you only calculate the accuracy and the model learned is discarded and never retained.

我想念什么?是否保留这样的模式?DQN与KNN的关系如何保留?

What am I missing? If such a model is retained? How is it retained and does the method differ for DQN from KNN?

推荐答案

K折交叉验证不会在每次迭代中重新训练模型.相反，它训练和评估K个不同的独立 (可以并行化)具有不同数据集折叠度的模型，但具有相同的折叠度超参数.这并不是要获得更准确的模型，而是要通过计算汇总的验证得分(即:您可以估计模型准确性的平均值和标准差)来获得更准确的(从统计意义上来说)验证.

K-fold cross-validation does not retrain models in each iteration. Instead, it trains and evaluates K different independent (could be parallelized) models with different folds of the dataset, but with the same hyper-parameters. This is not to get a more accurate model, but to get a more accurate (statistically speaking) validation by computing the aggregated validation score (i.e.: you can estimate the mean and stddev of the accuracy of the model).

然后，您可以只保留其中一个模型，并对其度量使用汇总估计(而不是使用针对该模型的特定折叠中计算出的一个)，或(从头开始)训练具有完整数据集的新模型.在后一种情况下，对模型指标的最佳估计仍然是先前的汇总指标，但是可以使用新的未使用的测试集来估算新的指标.那么，为什么要这么做呢?好吧，这是因为您通常将交叉验证与超参数调整结合使用.因此，每次调整超参数时，您只需使用交叉验证检查汇总的指标估计即可.但是，当您完成模型调整后，您将使用看不见的测试集计算最终指标.

You can then just keep one of those models and use the aggregated estimation for its metrics (instead of using the one computed in the specific fold for that model), or train (from scratch) a new model with the complete dataset. In this last case, your best estimation for the model metrics is still the previous aggregated metrics, but a new unused test set could be used to estimate a new metric. So, why would you do that? Well, that is because you normally use cross-validation combined with hyper-parameters tuning. So, every time you tune your hyperparameters, you just check the aggregated metric estimation with cross-validation. But, when you finish tuning your model, you compute the final metrics with the unseen test set.

这篇关于如何从k折交叉验证的每个折中学习?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！