numpy:点(a，b)和(a * b).sum()之间的差异 [英] Numpy: Difference between dot(a,b) and (a*b).sum()

问题描述

对于一维numpy数组，这两个表达式应该产生相同的结果(理论上):

For 1-D numpy arrays, this two expressions should yield the same result (theorically):

(a*b).sum()/a.sum()
dot(a, b)/a.sum()

后者使用dot()并且速度更快.但是，哪一个更准确?为什么?

The latter uses dot() and is faster. But which one is more accurate? Why?

某些上下文如下.

我想使用numpy计算样本的加权方差. 我在另一个答案中找到了dot()表达式，并加了一条评论，指出它应该更准确.但是，这里没有给出解释.

I wanted to compute the weighted variance of a sample using numpy. I found the dot() expression in another answer, with a comment stating that it should be more accurate. However no explanation is given there.

推荐答案

Numpy点是调用要在编译(或构建自己的)链接的BLAS库的例程之一.这样做的重要性在于BLAS库可以利用乘累加运算(通常是融合乘加)来限制计算执行的舍入次数.

Numpy dot is one of the routines that calls the BLAS library that you link on compile (or builds its own). The importance of this is the BLAS library can make use of Multiply–accumulate operations (usually Fused-Multiply Add) which limit the number of roundings that the computation performs.

请执行以下操作:

>>> a=np.ones(1000,dtype=np.float128)+1E-14 
>>> (a*a).sum()  
1000.0000000000199948
>>> np.dot(a,a)
1000.0000000000199948

不太精确，但足够接近.

Not exact, but close enough.

>>> a=np.ones(1000,dtype=np.float64)+1E-14
>>> np.dot(a,a)
1000.0000000000176  #off by 2.3948e-12
>>> (a*a).sum()
1000.0000000000059  #off by 1.40948e-11

np.dot(a, a)将是两者中更精确的，因为它使用的天数(a*a).sum()大约是浮点舍入次数的一半.

The np.dot(a, a) will be the more accurate of the two as it use approximately half the number of floating point roundings that the naive (a*a).sum() does.

Nvidia的一本书具有以下示例，表示4位精度. rn表示4舍入到最接近的4位数字:

A book by Nvidia has the following example for 4 digits of precision. rn stands for 4 round to the nearest 4 digits:

x = 1.0008
x2 = 1.00160064                    #    true value
rn(x2 − 1) = 1.6006 × 10−4         #    fused multiply-add
rn(rn(x2) − 1) = 1.6000 × 10−4     #    multiply, then add

当然，浮点数不会四舍五入到以10为底的小数点后16位，但是您知道了.

Of course floating point numbers are not rounded to the 16th decimal place in base 10, but you get the idea.

在上述符号中放置np.dot(a,a)并添加一些其他伪代码:

Placing np.dot(a,a) in the above notation with some additional pseudo code:

out=0
for x in a:
    out=rn(x*x+out)   #Fused multiply add

(a*a).sum()是:

arr=np.zeros(a.shape[0])   
for x in range(len(arr)):
    arr[x]=rn(a[x]*a[x])

out=0
for x in arr:
    out=rn(x+out)

由此很容易看出，与np.dot(a,a)相比，使用(a*a).sum()将数字舍入为两倍.这些小的差异相加可以立即改变答案.可以在此处找到其他示例.

From this its easy to see that the number is rounded twice as many times using (a*a).sum() compared to np.dot(a,a). These small differences summed can change the answer minutely. Additional exmaples can be found here.

这篇关于numpy:点(a，b)和(a * b).sum()之间的差异的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！