Tensorflow均方误差损失函数 [英] Tensorflow mean squared error loss function

问题描述

我在Tensorflow的回归模型的各个帖子中看到了一些不同的均方误差损失函数:

loss = tf.reduce_sum(tf.pow(prediction - Y,2))/(n_instances)
loss = tf.reduce_mean(tf.squared_difference(prediction, Y))
loss = tf.nn.l2_loss(prediction - Y)

两者之间有什么区别?

解决方案

我要说的是，第三个方程是不同的，而第一个和第二个方程在形式上是相同的，但是由于数值上的考虑，它们的行为有所不同.

我认为第三个方程(使用l2_loss)仅返回平方欧几里德范数的1/2，即输入的元素方平方之和，即x=prediction-Y.您没有将其除以任何地方的样本数量.因此，如果您有大量样本，则计算可能会溢出(返回Inf).

另两个在形式上是相同的，计算元素方平方的x张量的均值.但是，尽管文档中没有明确指定，但reduce_mean很可能使用一种适用于避免大量样本溢出的算法.换句话说，它可能不会尝试先将所有内容相加，然后然后除以N，而是使用某种滚动平均值，它可以适应任意数量的样本而不必引起溢出.

I have seen a few different mean squared error loss functions in various posts for regression models in Tensorflow:

loss = tf.reduce_sum(tf.pow(prediction - Y,2))/(n_instances)
loss = tf.reduce_mean(tf.squared_difference(prediction, Y))
loss = tf.nn.l2_loss(prediction - Y)

What are the differences between these?

解决方案

I would say that the third equation is different, while the 1st and 2nd are formally the same but behave differently due to numerical concerns.

I think that the 3rd equation (using l2_loss) is just returning 1/2 of the squared Euclidean norm, that is, the sum of the element-wise square of the input, which is x=prediction-Y. You are not dividing by the number of samples anywhere. Thus, if you have a very large number of samples, the computation may overflow (returning Inf).

The other two are formally the same, computing the mean of the element-wise squared x tensor. However, while the documentation does not specify it explicitly, it is very likely that reduce_mean uses an algorithm adapted to avoid overflowing with very large number of samples. In other words, it likely does not try to sum everything first and then divide by N, but use some kind of rolling mean that can adapt to an arbitrary number of samples without necessarily causing an overflow.

这篇关于Tensorflow均方误差损失函数的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！