地图或列表理解实现对通过循环调用函数的性能提升从哪里来的？ [英] Where does the performance boost of map or list comprehension implementations over calling a function over a loop come from?

问题描述

据我所知，在执行map时你可以更有效地利用内存，而不是通过循环来完成。但是，我发现使用map函数调用循环迭代的函数也有一定的提速。

这是来自存储记忆？我的意思是这样做的一个例子是以连续的方式放置内存。另外我可以看到，如果这些操作并行运行，那么速度也会提高，但我不认为这是事实。来自任何语言/软件包的map实现的任何已知优化示例都非常受欢迎！

- 编辑：我感觉好像我以前没有说明我的问题很好。

可能不完全公平比较。
例如，我测试循环实现，列表理解和映射函数。你会有什么想法让一个比另一个更快吗？不必要的蟒蛇;这更多的是关于在可迭代对象上应用函数的更有效算法的实现问题。一个有效的答案可能是通常对于每个地图/列表的理解风格的代码，你将能够比这更快地实现循环实现。或者在某些情况下，列表理解速度更快，但这与实现细节有关，这正是我所感兴趣的。

  import time 
 import numpy as np 
 
 
 def square2（x）：
 return x * x 
 
 
 def main（）：
 foobar = np.linspace（1,300,300）
 
 start_time = time.time（）
 res = [0] * len（foobar）
 for i，foo in enumerate（foobar）：
 res [i] = square2（foo）
 print（{} {} runtime seconds {}。format（ - * 8 ，time.time（） -  start_time， - * 8））
 
 res = [0] * len（foobar）
 start_time = time.time（）
 res = [square2（foo）for foo in foobar] 
 print（{} {} runtime seconds {}。format（ - * 8，time.time（） -  start_time， - * 8） ）
 
 start_time = time.time（）
 res = list（map（square2，foobar））
 print（{} {} runtime seconds {}。format  - * 8，time.time（） -  start_time， - * 8））
  

输出结果是：

  -------- 6.175041198730469e-05 runtime seconds -------- 
 -------- 5.984306335449219e-05运行时秒-------- 
 -------- 5.316734313964844e-05运行时秒-------- 
   
 
 
 进口时间
 def square（x）：
返回x * x 
 
 def test_loop（x）：
 res = [] 
 for i in x：
 res.append（square（i））
返回res 
 
 
 def test_map（x）：
返回列表（map（square，x））
 
 def test_map_local（x， square = square）：
 return list（map（square，x））
 
 
 def test_loop_local（x，square = square）：
 res = [] 
 for i in x：
 res.append（square（i））
 return res 
 
 def test_loop_local_local（x，square = square）：
 res = [] 
 append = res.append 
 for my x：
 append（square（i））
 return res 
 
 def test_comprehension（x ）：
 return [square（i）for i in x] 
 
 def test_comprehension_local（x，square = square）：
 return [square（i）for i in x] 
 
x =范围（1,10000000）
 
 start = time.time（）
 test_loop（x）
 stop = time.time（）
 print（Loop：，stop  -  start，seconds）
 
 start = time.time（）
 test_loop_local（x ）
 stop = time.time（）
 print（Loop-local：，stop  -  start，seconds）
 
 start = time.time（）
 test_loop_local_local（x）
 stop = time.time（）
 print（Loop-local-local：，stop  -  start，seconds）
 
 start = time.time（）
 test_map（x）
 stop = time.time（）
 print（Map：，stop  -  start，seconds）
 
 start = time.time（）
 test_map_local（x）
 stop = time.time（）
 print（Map-local：，stop  -  start，seconds）
 
 start = time.time（）
 test_comprehension（x）
 stop = time.time（）
 print（Comprehesion：，stop  -  start，seconds ）
 
 start = time.time（）
 test_comprehension_local（x）
 stop = time.time（）
 print（Comprehesion-local：，stop  - 开始，秒）
  
结果：
 
 
 循环：3.9749317169189453秒
本地循环：3.68653082847595 2秒
 Loop-local-local：3.006138563156128秒
地图：3.1068732738494873秒
地图本地：3.1318843364715576秒
 Comprehesion：2.973804235458374秒
 Comprehesion-local：2.7370445728302秒
  
因此，在本地map中查找函数并没有什么帮助，因为我期望因为地图在开始时只进行一次查询。真正令我感到惊讶的是，在理解和理解与本地功能之间似乎存在着不可忽视的差异。但不知道这是否是噪音。
 
I understand that you could be more efficient with memory in the implementation of map than in how you might do it over a loop. However, I see that using a map function over calling a function iterating over a loop has a speed boost as well.

Is this coming from some optimization in where to store the memory? One example of what I mean by this is placement of memory done in such a way that it is contiguous. Also I can see that if the operations were being in run in parallel then there would also be a speed boost, but I don't think this is the case. Examples from any known optimizations for map implementations from any language/package are very welcome!

-- Edit: I feel as if the example I had previously did not illustrate my question very well.

MAY NOT BE COMPLETELY FAIR COMPARISON.
For example, I test the a loop implementation, list comprehension, and the map function. Would you have any ideas of what would make one faster than the other? NOT NECESSARILY PYTHON; this is more a question about implementation of more efficient algorithms for applying a function over an iterable object. A valid answer might be that "typically for every map/list comprehension style of code, you would be able to make a loop implementation faster than that". Or for some case, the list comprehension is faster, but this relates to the implementation details and that is what I am interested in.
import time
import numpy as np


def square2(x):
return x*x


def main():
    foobar = np.linspace(1, 300, 300)

    start_time = time.time()
    res = [0] * len(foobar)
    for i, foo in enumerate(foobar):
        res[i] = square2(foo)
    print("{} {} runtime seconds {}".format("-"*8, time.time()-start_time, "-"*8))

    res = [0] * len(foobar)
    start_time = time.time()
    res = [square2(foo) for foo in foobar]
    print("{} {} runtime seconds {}".format("-"*8, time.time()-start_time, "-"*8))

    start_time = time.time()
    res = list(map(square2, foobar))
    print("{} {} runtime seconds {}".format("-"*8, time.time()-start_time, "-"*8))
The output is:
-------- 6.175041198730469e-05 runtime seconds --------
-------- 5.984306335449219e-05 runtime seconds --------
-------- 5.316734313964844e-05 runtime seconds --------

解决方案
So, the issue with function-calls in loops, in a dynamic language like Python, is that the interpreter has to evalute the refernces each time, and that is costly, especially for global variables. However, notice what happens when you make things local:
import time
def square(x):
    return x*x

def test_loop(x):
    res = []
    for i in x:
        res.append(square(i))
    return res


def test_map(x):
    return  list(map(square,x))

def test_map_local(x, square=square):
    return  list(map(square,x))


def test_loop_local(x, square=square):
    res = []
    for i in x:
        res.append(square(i))
    return res

def test_loop_local_local(x, square=square):
    res = []
    append = res.append
    for i in x:
        append(square(i))
    return res

def test_comprehension(x):
    return [square(i) for i in x]

def test_comprehension_local(x, square=square):
    return [square(i) for i in x]

x = range(1,10000000)

start = time.time()
test_loop(x)
stop = time.time()
print("Loop:", stop - start,"seconds")

start = time.time()
test_loop_local(x)
stop = time.time()
print("Loop-local:", stop - start, "seconds")

start = time.time()
test_loop_local_local(x)
stop = time.time()
print("Loop-local-local:", stop - start, "seconds")

start = time.time()
test_map(x)
stop = time.time()
print("Map:", stop - start, "seconds")

start = time.time()
test_map_local(x)
stop = time.time()
print("Map-local:", stop - start, "seconds")

start = time.time()
test_comprehension(x)
stop = time.time()
print("Comprehesion:", stop - start, "seconds")

start = time.time()
test_comprehension_local(x)
stop = time.time()
print("Comprehesion-local:", stop - start, "seconds")
Results:
Loop: 3.9749317169189453 seconds
Loop-local: 3.686530828475952 seconds
Loop-local-local: 3.006138563156128 seconds
Map: 3.1068732738494873 seconds
Map-local: 3.1318843364715576 seconds
Comprehesion: 2.973804235458374 seconds
Comprehesion-local: 2.7370445728302 seconds
So, making the function look-up in map local doesn't really help, as I would expect because map does a single look-up at the beginning. What really surprised me is that there seems to be a non-negligible difference between a comprehension and a comprehension with the function made local. Not sure if this is just noise, however.

                        
这篇关于地图或列表理解实现对通过循环调用函数的性能提升从哪里来的？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！