减少训练损失，稳定验证损失-模型是否过拟合? [英] Decreasing training loss, stable validation loss - is the model overfitting?

问题描述

我的模型是否过拟合?如果验证损失大量增加，而训练损失减少，我将确定它是否适合.但是，验证损失几乎是稳定的，因此我不确定.你能帮忙吗?

Does my model overfit? I would be sure it overfitted, if the validation loss increased heavily, while the training loss decreased. However the validation loss is nearly stable, so I am not sure. Can you please help?

推荐答案

• 我假设您使用的是不同的超参数?也许可以保存
  参数并使用一组不同的超参数恢复.
  此评论确实取决于您如何执行超参数
  优化.

• 尝试不同的训练/测试方式.这可能是特质的.

  • I assume that you're using different hyperparameters? Perhaps save
    the parameters and resume with a different set of hyperparameters.
    This comment really depends on how you're doing hyperparameter
    optimization.

  • Try with different training/test splits. It might be idiosyncratic. Especially with so few epochs.

    取决于训练模型和评估模型的成本，考虑将模型打包，类似于随机森林的运行方式.换句话说，让您的模型适合许多不同的训练/测试拆分，并以多数为标准对模型输出进行平均分类投票或预测概率的平均值.在这种情况下，我会偏于稍微过拟合的模型，因为平均可以减轻过度拟合的方式.但是我也不会训练到死亡，除非你非常适合许多神经网络，并以某种方式确保您将它们解相关类似于随机森林中随机子空间的方法.

    Depending on how costly it is to train the model and evaluate it, consider bagging your models, akin to how a random forest operates. In others words, fit your model to many different train/test splits, and average the model outputs, either in terms of a majority classification vote, or an averaging of the predicted probabilities. In this case, I'd err on the side of a slightly overfit model, because of the way that averaging can mitigate overfitting. But I wouldn't train to death either, unless you're going to fit very very many neural nets, and somehow ensure that you're decorrelating them akin to the method of random subspaces from random forests.

    这篇关于减少训练损失，稳定验证损失-模型是否过拟合?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！