如何在 TensorFlow 中对具有巨大正负值的数组/向量应用 softmax? [英] How to apply softmax on an array/vector with huge positive and negative values in TensorFlow?

问题描述

我在 TensorFlow 中使用 MNIST 数据集训练了一个卷积神经网络 (CNN).我从 MNIST 测试图像中计算每个图像的准确度，并寻找十个输出节点的值.我使用以下代码行来获取它(请参阅此处的所有代码:如何在 TensorFlow 中的 eval MNIST 测试数据期间从每个输出节点获取值?):

I train a convolutional neural network (CNN) with MNIST data set in TensorFlow. I calculate the accuracy for each image from the MNIST test images and looking for the values of the ten output-nodes. I use the following line of code to get it (see all code here: How to get the value from each output-node during eval MNIST testdata in TensorFlow?):

pred=prediction.eval(feed_dict={ x: testSet[0], y: testSet[1]})

这行代码的输出例如:

[[ -13423.92773438 -27312.79296875 20629.26367188 42987.953125
-34635.8203125 3714.84619141 -60946.6328125 106193.8828125
-20936.08789062 3940.52636719]]

当我尝试使用以下代码在此向量/数组上应用 tf.nn.softmax() 函数时:

When I try to apply the tf.nn.softmax() function at this vector/array with the following code:

pred_softmax = tf.nn.softmax(pred)
print(pred_softmax_new.eval())

我得到例如这个输出:

[[ 0. 0. 0. 0. 0. 0. 0. 1. 0. 0.]]

但我正在寻找这样的结果:

But I am looking for results like this:

[[ 0.001 0.207 0.001 0.001 0.002 0.001 0.001 0.9723 0.001 0.001]]

我测试这段代码:

test_a = tf.nn.softmax([31.0,23.0])
print(test_a.eval())

得到这个输出:

[ 9.99664664e-01 3.35350138e-04]

但是如果我增加这个值:

But if I increase the value like:

test_a = tf.nn.softmax([45631.0,65423.0])

我得到这个输出:

[ 0. 1.]

所以我的问题是有一种方法可以为十个输出节点获得良好的可读输出，例如:

So my question is there a way to get good readable outputs for the ten output-nodes like for example this:

[[ 0.001 0.207 0.001 0.001 0.002 0.001 0.001 0.9723 0.001 0.001]]

推荐答案

这似乎是提到的数值稳定性问题 这里.我认为链接中提到的答案对您的情况没有帮助，因为您的数字之间的差异非常大.基本上，您的模型似乎以 0.999999999 的概率预测了 7 的值......

This seems like an issue of numerical stability mentioned here. I don't think that the answer mentioned in the link will help in your case though, as the differences between your numbers are extremely large. Basically, it seems that your model predict the value of 7 with probability 0.999999999....

这篇关于如何在 TensorFlow 中对具有巨大正负值的数组/向量应用 softmax?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！