为什么需要softmax函数?为什么不进行简单的归一化? [英] Why is softmax function necessory? Why not simple normalization?

问题描述

我不熟悉深度学习，因此这可能是一个初学者的问题. 据我了解，多层感知器中的softmax函数负责归一化和分配每个类的概率. 如果是这样，为什么我们不使用简单的归一化呢?

I am not familiar with deep learning so this might be a beginner question. In my understanding, softmax function in Multi Layer Perceptrons is in charge of normalization and distributing probability for each class. If so, why don't we use the simple normalization?

比方说，我们得到一个向量x = (10 3 2 1) 应用softmax，输出将为y = (0.9986 0.0009 0.0003 0.0001).

Let's say, we get a vector x = (10 3 2 1) applying softmax, output will be y = (0.9986 0.0009 0.0003 0.0001).

应用简单规范化(将每个元素除以sum(16)) 输出将为y = (0.625 0.1875 0.125 0.166).

Applying simple normalization (dividing each elements by the sum(16)) output will be y = (0.625 0.1875 0.125 0.166).

似乎简单的归一化也可以分布概率. 那么，在输出层上使用softmax函数的好处是什么?

It seems like simple normalization could also distribute the probabilities. So, what is the advantage of using softmax function on the output layer?

推荐答案

规范化并不总是产生概率，例如，当您考虑负值时，它就不起作用.或如果值的总和为零怎么办?

Normalization does not always produce probabilities, for example, it doesn't work when you consider negative values. Or what if the sum of the values is zero?

但是使用logit的指数变化，从理论上讲它永远不会为零，并且可以将logit的整个范围映射到概率中.因此它是首选，因为它实际上可以工作.

But using exponential of the logits changes that, it is in theory never zero, and it can map the full range of the logits into probabilities. So it is preferred because it actually works.

这篇关于为什么需要softmax函数?为什么不进行简单的归一化?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！