逻辑/S型函数实现数值精度 [英] logistic / sigmoid function implementation numerical precision

问题描述

在scipy.special.expit中，物流功能的实现如下:

in scipy.special.expit, logistic function is implemented like the following:

if x < 0
    a = exp(x) 
    a / (1 + a) 
else 
    1 / (1 + exp(-x))

但是，我看到了其他简单实现的语言/框架的实现

However, I have seen implementations in other languages/frameworks that simply do

1 / (1 + exp(-x))

我想知道scipy版本实际上能带来多少好处.

I am wondering how much benefit the scipy version actually brings.

对于非常小的x，结果接近0.即使exp(-x)溢出到Inf，该结果也有效.

For very small x, the result approaches to 0. It works even if exp(-x) overflows to Inf.

推荐答案

这实际上只是为了保持稳定性-放入非常大的值可能会返回意外结果.

It's really just for stability - putting in values that are very large in magnitude might return unexpected results otherwise.

如果expit的实现方式与1 / (1 + exp(-x))相同，则将-710的值放入函数中将返回nan，而-709的值将接近于预期的零.这是因为exp(710)太大，无法翻倍.

If expit was implemented just as 1 / (1 + exp(-x)) then putting a value of -710 into the function would return nan, whereas -709 would give a value close to zero as expected. This is because exp(710) is too big to be a double.

代码中的分支只是意味着可以避免这种情况.

The branching in the code just means that this scenario is avoided.

另请参见有关堆栈溢出的问题和答案.

这篇关于逻辑/S型函数实现数值精度的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！