如何在Tensorflow DNNRegressor Estimator模型中解释损失函数? [英] How to interpret loss function in Tensorflow DNNRegressor Estimator model?

问题描述

我正在使用Tensorflow DNNRegressor Estimator模型制作神经网络.但是调用estimator.train()函数将给出如下输出:

I am using Tensorflow DNNRegressor Estimator model for making a neural network. But calling estimator.train() function is giving output as follows:

即我的损失函数在每个步骤中都变化很大.但是据我所知，我的损失函数应该减少而无需迭代.另外，找到附带的截图以获取Tensorboard Visualization的损失功能:

I.e. my loss function is varying a lot with every step. But as far as I know, my loss function should decrease with no of iterations. Also, find the attached screenshot for Tensorboard Visualisation for loss function:

我无法弄清的疑问是:

• 是整体损失函数值(到目前为止处理的每个步骤的综合损失)还是该步骤的损失值?
• 如果是该步骤的损失值，那么如何获得整体损失函数的值并查看其趋势，我认为应该随着迭代次数的增加而减小?据我所知，这是我们在训练数据集时应该考虑的价值.
• 如果这是总损失值，那么为什么波动那么大?我想念什么吗?

推荐答案

首先，让我指出

First of all, let me point out that tf.contrib.learn.DNNRegressor uses a linear regression head with mean_squared_loss, i.e. simple L2 loss.

是否为整体损失函数值(每个 步骤处理到现在为止)还是只是该步骤的损失值?

Whether it is overall loss function value (combined loss for every step processed till now) or just that step's loss value?

图表上的每个点是到目前为止学习后最后一步的损失函数的值.

Each point on a chart is the value of a loss function on the last step after learning so far.

如果是该步骤的损失值，那么如何获得总损失的值 功能并查看其趋势，我认为应该随着 增加迭代次数吗?

If it is that step's loss value, then how to get value of overall loss function and see its trend, which I feel should decrease with increasing no of iterations?

没有整体损失函数，可能是指图表说明每一步损失的变化情况.这正是张量板向您显示的内容.您说得对，它的趋势并非应有的下降趋势.这表明您的神经网络没有学习.

There's no overall loss function, probably you mean a chart how the loss changed after each step. That's exactly what tensorboard is showing to you. You are right, its trend is not downwards, as it should. This indicates that your neural network is not learning.

如果这是总体损失值，那么为什么波动如此之大?我想念什么吗?

If this is overall loss value, then why is it fluctuating so much? Am I missing something?

神经网络无法学习的常见原因是对超参数的选择不正确(尽管

A common reason for the neural network not learning is poor choice of hyperparameters (though there are many more mistakes you can possibly make). For example:

• 学习率太大
• 学习率过低也有可能，这意味着神经网络正在学习，但是学习速度非常慢，因此您看不到它.
• 权重初始化可能太大，请尝试减小它
• 批量大小可能也太大
• 您为输入内容传递了错误的标签
• 培训数据包含缺失值或未规范化
• ...

我通常要检查神经网络是否至少以某种方式起作用，这是将训练集减少到几个示例，并尝试使网络过拟合.这个实验非常快，因此我可以尝试各种学习率，初始化方差和其他参数来找到一个最佳位置.一旦有了稳定的，不断减少的损失图，我便继续进行更大的调整.

What I usually do to check if the neural network is at least somehow working is reduce the training set to few examples and try to overfit the network. This experiment is very fast, so I can try various learning rates, initialization variance and other parameters to find a sweet spot. Once I have a steady decreasing loss chart, I go on with a bigger set.

这篇关于如何在Tensorflow DNNRegressor Estimator模型中解释损失函数?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！