什么是logits，softmax和softmax_cross_entropy_with_logits? [英] What is logits, softmax and softmax_cross_entropy_with_logits?

问题描述

我正在查看tensorflow API文档此处.在tensorflow文档中，他们使用了名为logits的关键字.它是什么? API文档中的许多方法都将其写为

I was going through the tensorflow API docs here. In the tensorflow documentation, they used a keyword called logits. What is it? In a lot of methods in the API docs it is written like

tf.nn.softmax(logits, name=None)

如果写的是那些logits仅是Tensors，为什么还要使用其他名称，例如logits?

If what is written is those logits are only Tensors, why keeping a different name like logits?

另一件事是，我无法区分两种方法.他们是

Another thing is that there are two methods I could not differentiate. They were

tf.nn.softmax(logits, name=None)
tf.nn.softmax_cross_entropy_with_logits(logits, labels, name=None)

它们之间有什么区别?这些文档对我来说还不清楚.我知道tf.nn.softmax的作用.但是没有其他.一个例子将非常有帮助.

What are the differences between them? The docs are not clear to me. I know what tf.nn.softmax does. But not the other. An example will be really helpful.

推荐答案

登录仅表示该函数对早期图层的未缩放输出进行操作，并且用于理解单位的相对缩放是线性的.特别是，这意味着输入的总和可能不等于1，表示值是 not 概率(您可能输入的是5).

Logits simply means that the function operates on the unscaled output of earlier layers and that the relative scale to understand the units is linear. It means, in particular, the sum of the inputs may not equal 1, that the values are not probabilities (you might have an input of 5).

tf.nn.softmax仅产生将 softmax函数应用于输入张量的结果. softmax压缩"输入，以便sum(input) = 1:这是一种规范化的方法. softmax的输出形状与输入相同:它只是将值归一化.可以将softmax 的输出解释为概率.

tf.nn.softmax produces just the result of applying the softmax function to an input tensor. The softmax "squishes" the inputs so that sum(input) = 1: it's a way of normalizing. The shape of output of a softmax is the same as the input: it just normalizes the values. The outputs of softmax can be interpreted as probabilities.

a = tf.constant(np.array([[.1, .3, .5, .9]]))
print s.run(tf.nn.softmax(a))
[[ 0.16838508  0.205666    0.25120102  0.37474789]]

相比之下，tf.nn.softmax_cross_entropy_with_logits在应用softmax函数之后计算结果的交叉熵(但是以数学上更仔细的方式将其全部在一起).它类似于以下结果:

In contrast, tf.nn.softmax_cross_entropy_with_logits computes the cross entropy of the result after applying the softmax function (but it does it all together in a more mathematically careful way). It's similar to the result of:

sm = tf.nn.softmax(x)
ce = cross_entropy(sm)

交叉熵是一个汇总指标:对所有元素求和.在形状为[2,5]的张量上的tf.nn.softmax_cross_entropy_with_logits输出为形状为[2,1](将第一维视为批处理).

The cross entropy is a summary metric: it sums across the elements. The output of tf.nn.softmax_cross_entropy_with_logits on a shape [2,5] tensor is of shape [2,1] (the first dimension is treated as the batch).

如果要进行优化以最小化交叉熵，并且并且要在最后一层之后进行软最大化，则应使用tf.nn.softmax_cross_entropy_with_logits而不是自己进行，因为它涵盖了数值不稳定的拐角情况以数学上正确的方式.否则，您最终会在这里和那里添加少量epsilons来对其进行破解.

If you want to do optimization to minimize the cross entropy AND you're softmaxing after your last layer, you should use tf.nn.softmax_cross_entropy_with_logits instead of doing it yourself, because it covers numerically unstable corner cases in the mathematically right way. Otherwise, you'll end up hacking it by adding little epsilons here and there.

编辑于2016-02-07: 如果您具有单类标签，而一个对象只能属于一个类，则现在可以考虑使用tf.nn.sparse_softmax_cross_entropy_with_logits，这样就不必将标签转换为密集的一键热阵列.在0.6.0版本之后添加了此功能.

Edited 2016-02-07: If you have single-class labels, where an object can only belong to one class, you might now consider using tf.nn.sparse_softmax_cross_entropy_with_logits so that you don't have to convert your labels to a dense one-hot array. This function was added after release 0.6.0.

这篇关于什么是logits，softmax和softmax_cross_entropy_with_logits?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！