如何在Python中实现Softmax函数 [英] How to implement the Softmax function in Python

问题描述

在 Udacity的深度学习课程中，softmax y_i的值就是指数除以整个Y向量的指数之和:

From the Udacity's deep learning class, the softmax of y_i is simply the exponential divided by the sum of exponential of the whole Y vector:

其中S(y_i)是y_i的softmax函数，e是指数，j是no.输入向量Y中的列数.

Where S(y_i) is the softmax function of y_i and e is the exponential and j is the no. of columns in the input vector Y.

我尝试了以下操作:

import numpy as np

def softmax(x):
    """Compute softmax values for each sets of scores in x."""
    e_x = np.exp(x - np.max(x))
    return e_x / e_x.sum()

scores = [3.0, 1.0, 0.2]
print(softmax(scores))

返回:

[ 0.8360188   0.11314284  0.05083836]

但是建议的解决方案是:

But the suggested solution was:

def softmax(x):
    """Compute softmax values for each sets of scores in x."""
    return np.exp(x) / np.sum(np.exp(x), axis=0)

产生的输出与第一个实现相同，即使第一个实现显式地获取各列和最大值的差，然后除以总和.

which produces the same output as the first implementation, even though the first implementation explicitly takes the difference of each column and the max and then divides by the sum.

有人可以用数学方式说明为什么吗?一个正确而另一个错误吗?

在代码和时间复杂度方面，实现是否相似?哪个更有效率?

推荐答案

它们都是正确的，但是从数值稳定性的角度来看，您是首选.

They're both correct, but yours is preferred from the point of view of numerical stability.

您从

e ^ (x - max(x)) / sum(e^(x - max(x))

利用a ^(b-c)=(a ^ b)/(a ^ c)的事实

By using the fact that a^(b - c) = (a^b)/(a^c) we have

= e ^ x / (e ^ max(x) * sum(e ^ x / e ^ max(x)))

= e ^ x / sum(e ^ x)

另一个答案是什么.您可以将max(x)替换为任何变量，否则它将抵消.

Which is what the other answer says. You could replace max(x) with any variable and it would cancel out.

这篇关于如何在Python中实现Softmax函数的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！