如何知道是欠拟合还是过拟合? [英] How to know if underfitting or overfitting is occuring?
问题描述
我正在尝试使用两个类进行图像分类.我有 1000 张平衡类的图像.当我训练模型时，我得到的恒定验证准确度较低，但验证损失减少.这是过拟合还是欠拟合的迹象?我还应该注意，我正在尝试使用新类和不同的数据集重新训练 Inception V3 模型.
纪元1/102/2 [==============================]2/2 [================================] - 126s 63s/step - 损失:0.7212 - acc:0.5312 - val_loss:0.7981 - val_acc:0.3889时代 2/102/2 [==============================]2/2 [==============================] - 70 秒 35 秒/步 - 损失:0.6681 - acc:0.5959 - val_loss:0.7751 - val_acc:0.3889时代 3/102/2 [==============================]2/2 [================================] - 71s 35s/step - 损失:0.7313 - acc:0.4165 - val_loss:0.7535 - val_acc:0.3889时代 4/102/2 [==============================]2/2 [================================] - 67s 34s/step - 损失:0.6254 - acc:0.6603 - val_loss:0.7459 - val_acc:0.3889纪元 5/102/2 [==============================]2/2 [================================] - 68s 34s/step - 损失:0.6717 - acc:0.5959 - val_loss:0.7359 - val_acc:0.3889时代 6/102/2 [==============================]2/2 [================================] - 107s 53s/step - 损失:0.6633 - acc:0.5938 - val_loss:0.7259 - val_acc:0.3889时代 7/102/2 [==============================]2/2 [==============================] - 67s 33s/step - 损失:0.6674 - acc:0.6411 - val_loss:0.7160 - val_acc:0.3889时代 8/102/2 [==============================]2/2 [================================] - 105s 53s/step - 损失:0.6296 - acc:0.6562 - val_loss:0.7099 - val_acc:0.3889时代 9/102/2 [==============================]2/2 [==============================] - 67s 34s/step - 损失:0.5717 - acc:0.8273 - val_loss:0.7064 - val_acc:0.4444时代 10/102/2 [==============================]2/2 [==============================] - 103s 52s/step - 损失:0.6276 - acc:0.6875 - val_loss:0.7035 - val_acc:0.4444什么是过拟合
当模型对训练数据过于具体(或不够具体)并且不能很好地外推到真实领域时，就会发生过拟合(或欠拟合).从现在开始我只会说过度拟合以挽救我糟糕的打字手指[*]
我认为
显然，绿线，一个
虽然损失曲线有时更漂亮和对数，但请注意这里的趋势是训练误差仍在下降但测试误差在上升.这是过度拟合的一个很大的危险信号.
上面的图是过拟合的，和以前一样.下图不是.
<小时>这种情况什么时候发生?
当模型参数过多时，它很容易过拟合(就像 n 次多项式到 n-1 个点).同样，参数不足的模型也可能欠拟合.
某些正则化技术，例如 dropout 或批量标准化，或传统的 l-1 正则化战斗这个.我相信这超出了您的问题范围.
延伸阅读:
<小时>
脚注
[*] 没有理由继续写过拟合/欠拟合"，因为两者的推理是相同的，但显然指标被翻转了(一个决策边界没有足够地锁定真正的边界，如反对过于紧密地包裹在个别点上).一般来说，过拟合是更常见的避免，因为更多的迭代/更多的参数"是当前的主题.如果你有很多数据而不是很多参数，也许你真的担心欠拟合，但我对此表示怀疑.
[**] 将维基百科第一张图片中黑线比绿线更可取的想法形式化的一种方法是 在模型选择期间惩罚模型所需的参数数量
I'm trying to do image classification with two classes. I have 1000 images with balanced classes. When I train the model, I get a low constant validation accuracy but a decreasing validation loss. Is this a sign of overfitting or underfitting? I should also note that I'm attempting to retrain the Inception V3 model with new classes and a different dataset.
Epoch 1/10
2/2 [==============================]2/2 [==============================] - 126s 63s/step - loss: 0.7212 - acc: 0.5312 - val_loss: 0.7981 - val_acc: 0.3889
Epoch 2/10
2/2 [==============================]2/2 [==============================] - 70s 35s/step - loss: 0.6681 - acc: 0.5959 - val_loss: 0.7751 - val_acc: 0.3889
Epoch 3/10
2/2 [==============================]2/2 [==============================] - 71s 35s/step - loss: 0.7313 - acc: 0.4165 - val_loss: 0.7535 - val_acc: 0.3889
Epoch 4/10
2/2 [==============================]2/2 [==============================] - 67s 34s/step - loss: 0.6254 - acc: 0.6603 - val_loss: 0.7459 - val_acc: 0.3889
Epoch 5/10
2/2 [==============================]2/2 [==============================] - 68s 34s/step - loss: 0.6717 - acc: 0.5959 - val_loss: 0.7359 - val_acc: 0.3889
Epoch 6/10
2/2 [==============================]2/2 [==============================] - 107s 53s/step - loss: 0.6633 - acc: 0.5938 - val_loss: 0.7259 - val_acc: 0.3889
Epoch 7/10
2/2 [==============================]2/2 [==============================] - 67s 33s/step - loss: 0.6674 - acc: 0.6411 - val_loss: 0.7160 - val_acc: 0.3889
Epoch 8/10
2/2 [==============================]2/2 [==============================] - 105s 53s/step - loss: 0.6296 - acc: 0.6562 - val_loss: 0.7099 - val_acc: 0.3889
Epoch 9/10
2/2 [==============================]2/2 [==============================] - 67s 34s/step - loss: 0.5717 - acc: 0.8273 - val_loss: 0.7064 - val_acc: 0.4444
Epoch 10/10
2/2 [==============================]2/2 [==============================] - 103s 52s/step - loss: 0.6276 - acc: 0.6875 - val_loss: 0.7035 - val_acc: 0.4444
What is overfitting
Overfitting ( or underfitting) occurs when a model is too specific (or not specific enough) to the training data, and doesn't extrapolate well to the true domain. I'll just say overfitting from now on to save my poor typing fingers [*]
I think the wikipedia image is good:
Clearly, the green line, a decision boundary trying to separate the red class from the blue, is "overfit", because although it will do well on the training data, it lacks the "regularized" form we like to see when generalizing [**].
These CMU slides on overfitting/cross validation also make the problem clear:
And here's some more intuition for good measure
When does overfitting occur, generally?
Overfitting is observed numerically when the testing error does not reflect the training error
Obviously, the testing error will always (in expectation) be worse than the training error, but at a certain number of iterations, the loss in testing will start to increase, even as the loss in training continues to decline.
How to tell when a model has overfit visually?
Overfitting can be observed by plotting the decision boundary (as in the wikipedia image above) when dimensionality allows, or by looking at testing loss in addition to training loss during the fit procedure
You don't give us enough points to make these graphs, but here's an example (from someone asking a similar question) showing what those loss graphs would look like:
While loss curves are sometimes more pretty and logarthmic, note the trend here that training error is still decreasing but testing error is on the rise. That's a big red flag for overfitting. SO discusses loss curves here
The slightly cleaner and more real-life example is from this CMU lecture on ovefitting ANN's:
The top graph is overfitting, as before. The bottom graph is not.
When does this occur?
When a model has too many parameters, it is susceptible to overfitting (like a n-degree polynomial to n-1 points). Likewise, a model with not enough parameters can be underfit.
Certain regularization techniques like dropout or batch normalization, or traditionally l-1 regularization combat this. I believe this is beyond the scope of your question.
Further reading:
- A good statistics-SO question and answers
- Dense reading: bounds on overfitting with some models
- Lighter reading: general overview
- The related bias-variance tradeoff
Footnotes
[*] There's no reason to keep writing "overfitting/underfitting", since the reasoning is the same for both, but the indicators are flipped, obviously (a decision boundary that hasn't latched onto the true border enough, as opposed to being too tightly wrapped against individual points). In general, overfitting is the more common to avoid, since "more iterations/more parameters" is the current theme. If you have lots of data and not lot of parameters, maybe you really are worried about underfitting, but I doubt it.
[**] One way to formalize the idea that the black line is preferable than the green one in the first image from wikipedia is to penalize the number of parameters required by your model during model selection
这篇关于如何知道是欠拟合还是过拟合?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
