为什么列表理解比数组乘以numpy要快得多? [英] Why list comprehension is much faster than numpy for multiplying arrays?

问题描述

最近我回答了这个问题，该问题希望将2个列表相乘，一些用户建议使用以下方式使用numpy，我认为这是正确的方式:

Recently I answered to THIS question which wanted the multiplication of 2 lists,some user suggested the following way using numpy, alongside mine which I think is the proper way :

(a.T*b).T

我还发现aray.resize()具有相同的性能.无论如何，另一个答案提出了使用列表理解的解决方案:

Also I found that aray.resize() has a same performance like that. any way another answer suggested a solution using list comprehension :

[[m*n for n in second] for m, second in zip(b,a)]

但是在基准测试之后，我发现列表理解的执行速度比numpy快得多:

But after the benchmark I saw that the list comprehension performs very faster than numpy :

from timeit import timeit

s1="""
a=[[2,3,5],[3,6,2],[1,3,2]]
b=[4,2,1]

[[m*n for n in second] for m, second in zip(b,a)]
"""
s2="""
a=np.array([[2,3,5],[3,6,2],[1,3,2]])
b=np.array([4,2,1])

(a.T*b).T
"""

print ' first: ' ,timeit(stmt=s1, number=1000000)
print 'second : ',timeit(stmt=s2, number=1000000,setup="import numpy as np")

结果:

 first:  1.49778485298
second :  7.43547797203

如您所见，numpy的速度大约快5倍.但最令人惊讶的是，它无需使用转置和以下代码即可更快运行:

As you can see numpy is approximately 5 time faster. but most surprising thing was that its faster without using transpose, and for following code :

a=np.array([[2,3,5],[3,6,2],[1,3,2]])
b=np.array([[4],[2],[1]])

a*b 

列表理解仍然快了5倍.所以除了这点之外，这里的列表理解是在C语言中执行的，我们使用了2个嵌套循环和一个zip函数.那是什么原因呢?是否由于numpy中的*操作?

The list comprehension still was 5 time faster.So besides of this point that list comprehensions performs in C here we used 2 nested loop and a zip function So what can be the reason? Is it because of operation * in numpy?

还要注意，timeit没问题，在这里我将import部分放在了setup中.

Also note that there is no problem with timeit here I putted the import part in setup.

我也尝试过使用更大的arras，差异变小但仍然没有意义:

I also tried it with larger arras, the difference gets lower but still doesn't make sense :

s1="""
a=[[2,3,5],[3,6,2],[1,3,2]]*10000
b=[4,2,1]*10000

[[m*n for n in second] for m, second in zip(b,a)]
"""
s2="""
a=np.array([[2,3,5],[3,6,2],[1,3,2]]*10000)
b=np.array([4,2,1]*10000)

(a.T*b).T

"""



print ' first: ' ,timeit(stmt=s1, number=1000)
print 'second : ',timeit(stmt=s2, number=1000,setup="import numpy as np")

结果:

 first:  10.7480301857
second :  13.1278889179

推荐答案

创建numpy数组比创建列表要慢得多:

Creation of numpy arrays is much slower than creation of lists:

In [153]: %timeit a = [[2,3,5],[3,6,2],[1,3,2]]
1000000 loops, best of 3: 308 ns per loop

In [154]: %timeit a = np.array([[2,3,5],[3,6,2],[1,3,2]])
100000 loops, best of 3: 2.27 µs per loop

在肉之前，NumPy函数调用还可能产生固定成本 可以通过快速的基础C/Fortran函数执行计算.这可以包括确保输入为NumPy数组，

There can also fixed costs incurred by NumPy function calls before the meat of the calculation can be performed by a fast underlying C/Fortran function. This can include ensuring the inputs are NumPy arrays,

在假设NumPy之前需要牢记这些设置/固定成本 解决方案本质上比纯Python解决方案快. NumPy在何时闪耀 您设置大数组一次，然后执行许多快速NumPy操作 在阵列上.如果数组很小，它可能无法胜过纯Python 因为设置成本可能超过卸载计算的收益 编译C/Fortran函数.对于小阵列，可能根本不够 计算使其值得.

These setup/fixed costs are something to keep in mind before assuming NumPy solutions are inherently faster than pure-Python solutions. NumPy shines when you set up large arrays once and then perform many fast NumPy operations on the arrays. It may fail to outperform pure Python if the arrays are small because the setup cost can outweigh the benefit of offloading the calculations to compiled C/Fortran functions. For small arrays there simply may not be enough calculations to make it worth it.

如果稍微增加数组的大小，然后移动数组的创建 进入设置，那么NumPy可以比纯Python快得多:

If you increase the size of the arrays a bit, and move creation of the arrays into the setup, then NumPy can be much faster than pure Python:

import numpy as np
from timeit import timeit

N, M = 300, 300

a = np.random.randint(100, size=(N,M))
b = np.random.randint(100, size=(N,))

a2 = a.tolist()
b2 = b.tolist()

s1="""
[[m*n for n in second] for m, second in zip(b2,a2)]
"""

s2 = """
(a.T*b).T
"""

s3 = """
a*b[:,None]
"""

assert np.allclose([[m*n for n in second] for m, second in zip(b2,a2)], (a.T*b).T)
assert np.allclose([[m*n for n in second] for m, second in zip(b2,a2)], a*b[:,None])

print 's1: {:.4f}'.format(
    timeit(stmt=s1, number=10**3, setup='from __main__ import a2,b2'))
print 's2: {:.4f}'.format(
    timeit(stmt=s2, number=10**3, setup='from __main__ import a,b'))
print 's3: {:.4f}'.format(
    timeit(stmt=s3, number=10**3, setup='from __main__ import a,b'))

收益

s1: 4.6990
s2: 0.1224
s3: 0.1234

这篇关于为什么列表理解比数组乘以numpy要快得多?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！