测试结果是否可能比验证结果更好？ [英] Would it be possible that test result is better than validation result?

问题描述

我正在训练CNN进行分类任务，然后将数据集分为三部分：70％用于训练，15％用于验证和最后15％用于测试。我使用训练集来训练网络，并使用验证集来选择超参数。完成所有工作后，我使用测试集测试了我的模型，但是事实证明，该模型在测试集上的表现要好于验证集（一个获得85％的准确性，另一个获得80％的准确性）。

I'm training a CNN for a classification task and I split the dataset into three parts: 70% for training, 15% for validation and final 15% for testing. I use the training set to train the network and use the validation set to choose the hyper-parameters. After all the works, I tested my model using the test set, however, it turned out the model performed better on the test set than the validation set(one got 85% accuracy and the other got 80% accuracy).

是否可能或者我做错了什么？由于我花了很多精力在验证集上提高模型性能，而测试集在模型训练的整个过程中都是不可见的。

Would it be possible or I did something wrong? Since I put much effort on improving the model performance on validation set while the test set is invisible during the whole process of model training.

推荐答案

当测试集比验证更好地表示训练数据时，这是可能的。通常，这意味着您拆分数据的方式存在一些问题

It is possible when your test set is a better representation of your training data than validation. Usually it means there is some problem is the way you split the data

尝试在随机整理数据后重复实验，并再次创建拆分以确保它是不是由于幸运的分裂。创建拆分时是否使用了分层抽样。

Try repeating the experiment after randomly shuffling the data and by creating the splits again to make sure that it is not due to the lucky splits. Have you used stratified sampling when you have created the splits.

这篇关于测试结果是否可能比验证结果更好？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！