tensorflow中sparse_softmax_cross_entropy_with_logits函数的原点编码在哪里 [英] Where is the origin coding of sparse_softmax_cross_entropy_with_logits function in tensorflow

问题描述

我想知道tensorflow函数是什么sparse_softmax_cross_entropy_with_logits 在数学上正是这样做的.但我找不到编码的来源.你能帮我吗?

I want to know what the tensorflow function sparse_softmax_cross_entropy_with_logits mathematically is exactly doing. But I can't find the origin of the coding. Can you help me?

推荐答案

sparse_softmax_cross_entropy_with_logits 相当于以下的数值稳定版本:

sparse_softmax_cross_entropy_with_logits is equivalent to a numerically stable version of the following:

<代码>-1.* tf.gather(tf.log(tf.nn.softmax(logits)), 目标)

或者，在更可读"的 numpy 代码中:

or, in more "readable" numpy-code:

<代码>-1.* np.log(softmax(logits))[目标]

其中 softmax(x) = np.exp(x)/np.sum(np.exp(x)).

也就是说，它计算提供的 logits 的 softmax，取其 log 来检索 log-probabilities，并将 log-probabilities 切片以检索目标的 log-probability.

That is, it computes the softmax of the provided logits, takes the log thereof to retrieve the log-probabilities, and slices the log-probabilities to retrieve the log-probability of the target.

但是，它通过向某些操作添加小值以数值稳定的方式(此处可能会出现一些问题)来实现这一点.这意味着计算上述 - 详细 - 版本只会产生与 nn.sparse_softmax_cross_entropy_with_logits 相同的值(运行一些测试表明差异始终小于 2e-6).

However, it does so in a numerically stable way (a couple of things can go wrong here) by adding small values to some of the operations. This means that computing the above - verbose - version will only approximately result in the same values as nn.sparse_softmax_cross_entropy_with_logits (running some tests showed that the difference is consistently smaller than 2e-6).

这篇关于tensorflow中sparse_softmax_cross_entropy_with_logits函数的原点编码在哪里的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！