为什么我的神经网络永远不会过拟合? [英] Why does my neural network never overfit?

问题描述

我正在训练带有10个隐藏层的深层残差网络，其中包含游戏数据.

I am training a deep residual network with 10 hidden layers with game data.

有人知道为什么我在这里没有过拟合吗? 经过100次训练后，训练和测验损失仍在减少.

Does anyone have an idea why I don't get any overfitting here? Training and test loss still decreasing after 100 epochs of training.

https://imgur.com/Tf3DIZL

推荐答案

仅几个建议:

1. 对于深度学习，建议甚至进行90/10或95/5拆分(Andrew Ng)
2. 曲线之间的微小差异意味着您的learning_rate没有被调整；尝试增加它(如果您要实现某种智能" lr-reduce，则可能要增加epochs的数量)
3. DNN尝试对少量数据(10-100行)和大量迭代进行过拟合也是合理的
4. 检查集合中是否有数据泄漏:每层内部的权重分析可能会帮助您

1. for deep learning is recommended to do even 90/10 or 95/5 splitting (Andrew Ng)
2. this small difference between curves means that your learning_rate is not tuned; try to increase it (and, probably, number of epochs if you will implement some kind of 'smart' lr-reduce)
3. it is also reasonable for DNN to try to overfit with the small amount of data (10-100 rows) and an enormous number of iterations
4. check for data leakage in the set: weights analysis inside each layer may help you in this

这篇关于为什么我的神经网络永远不会过拟合?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！