为什么numpy ndarray比简单循环的列表慢得多? [英] Why is numpy ndarray much slower than lists for simple loops?

问题描述

我是Python的新手，所以我决定解决一些常见的挑战，以提高我的语言知识.我了解了numpy及其有效的ndarrays，因此尝试了以下实验:

I'm new to Python so I've decided to solve some common challenges to improve my knowledge of the language. I learned about numpy and its efficient ndarrays so I attempted the following experiment:

考虑2和问题(例如，此处)，然后解决它天真的方式(此问题的目的无关紧要). 这是使用python列表的解决方案:

Consider the 2 sum problem (e.g. here) and let's solve it the naive way (it doesn't matter for the purpose of this question). Here's a solution with python's lists:

from  itertools import combinations

def twosum1(n_lst):
    pairs=list(combinations(n_lst,2))
    solutions=[]
    for pair in pairs:
        if sum(pair)==7: solutions.append(pair)
    return(solutions)

然后我使用np.arrays创建了一个版本，希望它可以大大加快计算速度:

Then I created a version using np.arrays expecting it will drastically speed up the calculation:

from  itertools import combinations
import numpy as np

def twosum2(n_lst):
    pairs=np.array(list(combinations(n_lst,2)),dtype=int)
    return pairs[pairs[:,1]+pairs[:,0]==7]

但是，在对两个函数进行计时之后，twosum2比twosum1慢大约2倍.因此，我认为问题可能出在元素的动态选择上，因此我用ndarrays替换了列表，从而编写了twosum1的精确副本...

However, after timing the two functions, twosum2 is about 2x slower than twosum1. So I thought that the problem maybe in the dynamical selection of elements, so I've written an exact copy of twosum1 by replacing lists with ndarrays ...

def twosum3(n_lst):
    pairs=np.array(list(combinations(n_lst,2)))
    solutions=np.empty((0,2))
    for pair in pairs:
        if np.sum(pair)==7: 
            solutions=np.append(solutions,[pair],axis=0)
    return(solutions)

...并且生成的功能比原始功能慢10倍！

... and the resulting function was 10x slower than the original!

这怎么可能?我在这里做错了什么?显然，删除循环并用ndarrays替换列表不足以提高速度(与我阅读此).

How is this possible? What I'm I doing wrong here? Clearly, removing loops and replacing lists with ndarrays is not enough to gain speed (contrary to what I learned reading this).

• 我在jupyter中使用％timeit来计时功能.
• 我为所有要使用的功能采用相同的基准.
• 我在3个函数中以相同的方式计算组合的事实告诉我，减速是由于numpy引起的，但是看不到如何实现.

推荐答案

代价高昂的操作是np.array(list(combinations(n_lst,2)),dtype=int)，因为python必须扫描列表中的每个成员，检查成员是否与int兼容，将其转换为整数并将其存储在数组中.

The costly operation is np.array(list(combinations(n_lst,2)),dtype=int) because python must scan each member of the list, check if member is 'int compatible', convert it in integer and store it in the array.

要达到numpy性能，必须在numpy中构思所有算法.例如:

To reach numpy performance, you must conceive all the algorithm in numpy. For example :

In [63]: n_lst=list(range(100))

In [64]: %timeit twosum1(n_lst)
11.2 ms ± 1.64 ms per loop (mean ± std. dev. of 7 runs, 100 loops each)


In [65]: np.vstack(np.where(np.add.outer(n_lst,n_lst)==7)).T
Out[65]: 
array([[0, 7],
       [1, 6],
       [2, 5],
       [3, 4],
       [4, 3],
       [5, 2],
       [6, 1],
       [7, 0]], dtype=int64)

In [66]: %timeit np.vstack(np.where(np.add.outer(n_lst,n_lst)==7)).T
306 µs ± 19 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

通过这种方式，您将赢得30到100的系数，具体取决于问题.

This way you will win a 30 to 100 factor , depending of the problem.

这篇关于为什么numpy ndarray比简单循环的列表慢得多?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！