如何知道是否发生欠拟合或过度拟合? [英] How to know if underfitting or overfitting is occuring?
问题描述
我正在尝试使用两个类对图像进行分类.我有1000张图片，均衡的班级.当我训练模型时，我得到的常数验证准确性较低，但是验证损失却越来越小.这是过度拟合或不足拟合的标志吗?我还应该注意，我正在尝试使用新的类和不同的数据集重新训练Inception V3模型.
I'm trying to do image classification with two classes. I have 1000 images with balanced classes. When I train the model, I get a low constant validation accuracy but a decreasing validation loss. Is this a sign of overfitting or underfitting? I should also note that I'm attempting to retrain the Inception V3 model with new classes and a different dataset.
Epoch 1/10
2/2 [==============================]2/2 [==============================] - 126s 63s/step - loss: 0.7212 - acc: 0.5312 - val_loss: 0.7981 - val_acc: 0.3889
Epoch 2/10
2/2 [==============================]2/2 [==============================] - 70s 35s/step - loss: 0.6681 - acc: 0.5959 - val_loss: 0.7751 - val_acc: 0.3889
Epoch 3/10
2/2 [==============================]2/2 [==============================] - 71s 35s/step - loss: 0.7313 - acc: 0.4165 - val_loss: 0.7535 - val_acc: 0.3889
Epoch 4/10
2/2 [==============================]2/2 [==============================] - 67s 34s/step - loss: 0.6254 - acc: 0.6603 - val_loss: 0.7459 - val_acc: 0.3889
Epoch 5/10
2/2 [==============================]2/2 [==============================] - 68s 34s/step - loss: 0.6717 - acc: 0.5959 - val_loss: 0.7359 - val_acc: 0.3889
Epoch 6/10
2/2 [==============================]2/2 [==============================] - 107s 53s/step - loss: 0.6633 - acc: 0.5938 - val_loss: 0.7259 - val_acc: 0.3889
Epoch 7/10
2/2 [==============================]2/2 [==============================] - 67s 33s/step - loss: 0.6674 - acc: 0.6411 - val_loss: 0.7160 - val_acc: 0.3889
Epoch 8/10
2/2 [==============================]2/2 [==============================] - 105s 53s/step - loss: 0.6296 - acc: 0.6562 - val_loss: 0.7099 - val_acc: 0.3889
Epoch 9/10
2/2 [==============================]2/2 [==============================] - 67s 34s/step - loss: 0.5717 - acc: 0.8273 - val_loss: 0.7064 - val_acc: 0.4444
Epoch 10/10
2/2 [==============================]2/2 [==============================] - 103s 52s/step - loss: 0.6276 - acc: 0.6875 - val_loss: 0.7035 - val_acc: 0.4444
推荐答案
什么是过度拟合
当模型对训练数据过于具体(或不够具体)而无法很好地推断出真实域时，就会发生过度拟合(或过度拟合).我只是说从现在开始过拟合，以节省我打字不佳的手指[*]
What is overfitting
Overfitting ( or underfitting) occurs when a model is too specific (or not specific enough) to the training data, and doesn't extrapolate well to the true domain. I'll just say overfitting from now on to save my poor typing fingers [*]
我认为维基百科图片很好:
很明显，绿线是决策边界将红色类与蓝色类分开，是过度拟合"，因为尽管它在训练数据上效果很好，但缺少"已正规化形式，我们希望看到一般化 [**].
Clearly, the green line, a decision boundary trying to separate the red class from the blue, is "overfit", because although it will do well on the training data, it lacks the "regularized" form we like to see when generalizing [**].
这些CMU幻灯片关于过度拟合/
These CMU slides on overfitting/cross validation also make the problem clear:
当测试误差不能反映训练误差时，会出现数值过大的拟合现象
Overfitting is observed numerically when the testing error does not reflect the training error
很明显，测试误差将总是(预期)比训练误差更严重，但是在一定的迭代次数下，即使训练的损失继续减少，测试的损失也会开始增加.
Obviously, the testing error will always (in expectation) be worse than the training error, but at a certain number of iterations, the loss in testing will start to increase, even as the loss in training continues to decline.
可以通过绘制决策边界来观察过度拟合(如 当维数允许时，或通过查看 在训练过程中除了训练损失外，还处于测试损失中
Overfitting can be observed by plotting the decision boundary (as in the wikipedia image above) when dimensionality allows, or by looking at testing loss in addition to training loss during the fit procedure
You don't give us enough points to make these graphs, but here's an example (from someone asking a similar question) showing what those loss graphs would look like:
尽管损失曲线有时更漂亮，更对数，但请注意此处的趋势，训练误差仍在减少，而测试误差却在上升.这是过度拟合的大警示. SO在此处讨论了损耗曲线
While loss curves are sometimes more pretty and logarthmic, note the trend here that training error is still decreasing but testing error is on the rise. That's a big red flag for overfitting. SO discusses loss curves here
The slightly cleaner and more real-life example is from this CMU lecture on ovefitting ANN's:
顶部图形与以往一样过拟合.底部图不是.
The top graph is overfitting, as before. The bottom graph is not.
何时发生?
当模型中的参数过多时，它很容易过度拟合(例如，对n-1个点的n次多项式).同样，参数不足的模型可能会欠拟合.
When a model has too many parameters, it is susceptible to overfitting (like a n-degree polynomial to n-1 points). Likewise, a model with not enough parameters can be underfit.
某些正则化技术，例如辍学或批正则化，或者传统上是l-1正则化打击这个.我认为这超出了您的问题范围.
Certain regularization techniques like dropout or batch normalization, or traditionally l-1 regularization combat this. I believe this is beyond the scope of your question.
- A good statistics-SO question and answers
- Dense reading: bounds on overfitting with some models
- Lighter reading: general overview
- The related bias-variance tradeoff
脚注
[*]没有理由继续写过度拟合/不足拟合"，因为两者的推理都相同，但是指标显然是翻转的(决策边界尚未充分锁定在真实边界上，因为反对太紧紧地包裹在个别观点上).通常，过度拟合是最常见的避免方法，因为更多的迭代/更多的参数"是当前的主题.如果您有大量数据而没有很多参数，也许您确实担心欠拟合，但我对此表示怀疑.
[*] There's no reason to keep writing "overfitting/underfitting", since the reasoning is the same for both, but the indicators are flipped, obviously (a decision boundary that hasn't latched onto the true border enough, as opposed to being too tightly wrapped against individual points). In general, overfitting is the more common to avoid, since "more iterations/more parameters" is the current theme. If you have lots of data and not lot of parameters, maybe you really are worried about underfitting, but I doubt it.
[**]在Wikipedia的第一张图片中，将黑线比绿线更可取的一种形式的形式是在模型选择过程中惩罚模型所需的参数数量
[**] One way to formalize the idea that the black line is preferable than the green one in the first image from wikipedia is to penalize the number of parameters required by your model during model selection
这篇关于如何知道是否发生欠拟合或过度拟合?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
