TensorFlow:对于交叉熵函数，我的 logits 格式是否正确? [英] TensorFlow: Are my logits in the right format for cross entropy function?

问题描述

好的，我准备运行

据我所知，logits 被插入到等式的右侧.也就是说，它们是每个 x(图像)的 q.如果它们是从 0 到 1 的概率……那对我来说很有意义.但是当我运行我的代码并以 logits 张量结束时，我没有得到概率.相反，我得到了正面和负面的浮动:

<预> <代码> -0.07264724 -0.15262917 0.06612295 ...，-0.03235611 0.08587133 0.01897052 0.04655019 -0.20552202 0.08725972 ...，-0.02107313 -0.00567073 0.03241089 0.06872301 -0.20756687 0.01094618 ...等

所以我的问题是……是吗?我是否必须以某种方式计算所有对数并将它们转换为从 0 到 1 的概率?

解决方案

需要注意的关键是 tf.nn.softmax_cross_entropy_with_logits(logits, labels) 对 logits 的每一行执行内部 softmax，因此在它们被输入到交叉熵方程之前，它们可以解释为概率.

因此，logits"不必是概率(甚至顾名思义，甚至是真正的对数概率)，因为在该操作中发生了内部规范化.

另一种写法:

xent = tf.nn.softmax_cross_entropy_with_logits(logits,labels)

...应该是:

softmax = tf.nn.softmax(logits)xent = -tf.reduce_sum(labels * tf.log(softmax), 1)

然而，这种替代方案将 (i) 数值稳定性较差(因为 softmax 可能计算出更大的值)和 (ii) 效率较低(因为在反向传播中会发生一些冗余计算).对于实际使用，我们建议您使用 tf.nn.softmax_cross_entropy_with_logits().

Alright, so I'm getting ready to run the tf.nn.softmax_cross_entropy_with_logits() function in Tensorflow.

It's my understanding that the 'logits' should be a Tensor of probabilities, each one corresponding to a certain pixel's probability that it is part of an image that will ultimately be a "dog" or a "truck" or whatever... a finite number of things.

These logits will get plugged into this cross entropy equation:

As I understand it, the logits are plugged into the right side of the equation. That is, they are the q of every x (image). If they were probabilities from 0 to 1... that would make sense to me. But when I'm running my code and ending up with a tensor of logits, I'm not getting probabilities. Instead I'm getting floats that are both positive and negative:

-0.07264724 -0.15262917  0.06612295 ..., -0.03235611  0.08587133 0.01897052 0.04655019 -0.20552202  0.08725972 ..., -0.02107313 -0.00567073 0.03241089 0.06872301 -0.20756687  0.01094618 ...,   etc

So my question is... is that right? Do I have to somehow calculate all my logits and turn them into probabilities from 0 to 1?

解决方案

The crucial thing to note is that tf.nn.softmax_cross_entropy_with_logits(logits, labels) performs an internal softmax on each row of logits so that they are interpretable as probabilities before they are fed to the cross entropy equation.

Therefore, the "logits" need not be probabilities (or even true log probabilities, as the name would suggest), because of the internal normalization that happens within that op.

An alternative way to write:

xent = tf.nn.softmax_cross_entropy_with_logits(logits, labels)

...would be:

softmax = tf.nn.softmax(logits)
xent = -tf.reduce_sum(labels * tf.log(softmax), 1)

However, this alternative would be (i) less numerically stable (since the softmax may compute much larger values) and (ii) less efficient (since some redundant computation would happen in the backprop). For real uses, we recommend that you use tf.nn.softmax_cross_entropy_with_logits().

这篇关于TensorFlow:对于交叉熵函数，我的 logits 格式是否正确?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！