tensorflow CNN丢失功能在张量板上上下波动(振荡)，如何将其移除? [英] tensorflow CNN loss function goes up and down ( oscilating) in tensorboard,How to remove them?

问题描述

我正在使用tensorflow在Audioset2017数据集上训练ResNet50 在训练和验证结果期间，我的损失函数起伏不定，总体趋势正在下降，但我对此感到恐惧.

I am training a ResNet50 on Audioset2017 dataset,with tensorflow during training and validating results,my loss function fluctuating,the overall trend is going down,but I am afraid of this.

我已经运行了100个纪元，批处理大小为100. 并降低和提高了学习率，但没有效果.

I have run with 100 epochs,with batch size=100. and have decrease and increase the learning rate,but had no effect.

想知道我的训练是正确的，我可以使用此网络吗?否则会导致错误的结果.我可以用一些技巧将其删除吗? 这些是我的训练和验证(评估)损失以及其他指标图片(来自张量板).

want to know is my training correct,can I use this network?or it causes wrong results.Can I remove them with some tricks? these are my train and validation(eval) loss and other metrics pictures(from tensorboard).

验证方式:

火车模式:

推荐答案

似乎在经过12k步后，模型开始过拟合.训练损失进一步减少，而验证损失(泛化误差)再次缓慢增加.此后，训练模型只会使情况变得更糟.

It seems like after 12k steps, the model starts to overfit. The training loss further decreases while the validation loss (generalization error) slowly increases again. After this point, training the model only makes it worse.

在下图中，您位于过度拟合区域.

In the figure below you are in the overfitting zone.

(来自 www.deeplearningbook.org )

您可能想通过增加正则化来降低模型对训练数据过度拟合的能力.例如，L2权重正则化或辍学.

You might want to reduce the model's ability to overfit on the training data by increasing regularization. For example, L2 weights regularization or dropout.

至于振荡.考虑到您的批次大小为100，它们很自然.

As for the oscillations. They are probably natural, given your batch size of 100.

这篇关于tensorflow CNN丢失功能在张量板上上下波动(振荡)，如何将其移除?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！