Tensorflow：损失减少，但精度稳定 [英] Tensorflow: loss decreasing, but accuracy stable
问题描述
我的团队正在Tensorflow中训练CNN，以对损坏/可接受的零件进行二进制分类。我们通过修改cifar10示例代码创建了代码。根据我以前在神经网络方面的经验，我一直训练直到损失非常接近0（远低于1）。但是，我们现在在训练期间（在单独的GPU上）使用验证集评估模型，似乎精度在大约6.7k步后停止增加，而损失在超过40k步后仍在稳步下降。这是由于过度拟合吗？一旦损失非常接近零，我们是否应该期望看到准确性的另一个峰值？当前的最大精度是不可接受的。我们应该杀死它并继续调整吗？你有什么建议吗？这是我们修改后的培训过程代码和图形。
My team is training a CNN in Tensorflow for binary classification of damaged/acceptable parts. We created our code by modifying the cifar10 example code. In my prior experience with Neural Networks, I always trained until the loss was very close to 0 (well below 1). However, we are now evaluating our model with a validation set during training (on a separate GPU), and it seems like the precision stopped increasing after about 6.7k steps, while the loss is still dropping steadily after over 40k steps. Is this due to overfitting? Should we expect to see another spike in accuracy once the loss is very close to zero? The current max accuracy is not acceptable. Should we kill it and keep tuning? What do you recommend? Here is our modified code and graphs of the training process.
https://gist.github.com/justineyster/6226535a8ee3f567e759c2ff2ae3776b
推荐答案
二进制交叉熵损失的减少不会意味着准确性的提高。考虑标签1，在时间步1、2、3和分类阈值0.5的预测0.2、0.4和0.6。时间步骤1和2将减少损失，但准确性不会增加。
A decrease in binary cross-entropy loss does not imply an increase in accuracy. Consider label 1, predictions 0.2, 0.4 and 0.6 at timesteps 1, 2, 3 and classification threshold 0.5. timesteps 1 and 2 will produce a decrease in loss but no increase in accuracy.
通过过度拟合训练数据来确保模型具有足够的容量。如果模型过度拟合训练数据，请使用遗漏，L1和L2正则化以及数据扩充等正则化技术避免过度拟合。
Ensure that your model has enough capacity by overfitting the training data. If the model is overfitting the training data, avoid overfitting by using regularization techniques such as dropout, L1 and L2 regularization and data augmentation.
最后，确认您的验证数据和训练数据来自同一分布。
Last, confirm your validation data and training data come from the same distribution.
这篇关于Tensorflow：损失减少，但精度稳定的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
