Python 3:为range(N)中的i创建[func(i)]列表理解的最有效方法 [英] Python 3: Most efficient way to create a [func(i) for i in range(N)] list comprehension
问题描述
说我有一个函数func(i)，它为整数i创建一个对象，而N是一些非负整数.那么创建等于此列表的列表(而不是范围)的最快方法是什么
Say I have a function func(i) that creates an object for an integer i, and N is some nonnegative integer. Then what's the fastest way to create a list (not a range) equal to this list
mylist = [func(i) for i in range(N)]
是否不使用高级方法(例如在C中创建函数)?我对上述列表理解的主要担心是，我不确定python是否事先知道range(N)的长度来预分配mylist，因此必须递增地重新分配列表.是这种情况还是python足够聪明，可以先将mylist分配为长度N，然后计算其元素?如果没有，创建mylist的最佳方法是什么?也许是这样吗?
without resorting to advanced methods like creating a function in C? My main concern with the above list comprehension is that I'm not sure if python knows beforehand the length of range(N) to preallocate mylist, and therefore has to incrementally reallocate the list. Is that the case or is python clever enough to allocate mylist to length N first and then compute it's elements? If not, what's the best way to create mylist? Maybe this?
mylist = [None]*N
for i in range(N): mylist[i] = func(i)
重新从先前的编辑中删除了误导性信息.
RE- Removed misleading information from a previous edit.
推荐答案
有人写道:""Python足够聪明.只要要迭代的对象具有__len__或__length_hint__方法，Python将调用它来确定大小并预分配数组.""
Somebody wrote: """Python is smart enough. As long as the object you're iterating over has a __len__ or __length_hint__ method, Python will call it to determine the size and preallocate the array."""
据我所知，列表理解中没有预分配. Python无法通过INPUT的大小来判断OUTPUT的大小.
As far as I can tell, there is no preallocation in a list comprehension. Python has no way of telling from the size of the INPUT what the size of the OUTPUT will be.
看下面的Python 2.6代码:
Look at this Python 2.6 code:
>>> def foo(func, iterable):
... return [func(i) for i in iterable]
...
>>> import dis; dis.dis(foo)
2 0 BUILD_LIST 0 #### build empty list
3 DUP_TOP
4 STORE_FAST 2 (_[1])
7 LOAD_FAST 1 (iterable)
10 GET_ITER
>> 11 FOR_ITER 19 (to 33)
14 STORE_FAST 3 (i)
17 LOAD_FAST 2 (_[1])
20 LOAD_FAST 0 (func)
23 LOAD_FAST 3 (i)
26 CALL_FUNCTION 1
29 LIST_APPEND #### stack[-2].append(stack[-1]); pop()
30 JUMP_ABSOLUTE 11
>> 33 DELETE_FAST 2 (_[1])
36 RETURN_VALUE
它只是构建一个空列表，并附加所有迭代传递的内容.
It just builds an empty list, and appends whatever the iteration delivers.
现在看一下这段代码，该代码在列表理解中有一个"if":
Now look at this code, which has an 'if' in the list comprehension:
>>> def bar(func, iterable):
... return [func(i) for i in iterable if i]
...
>>> import dis; dis.dis(bar)
2 0 BUILD_LIST 0
3 DUP_TOP
4 STORE_FAST 2 (_[1])
7 LOAD_FAST 1 (iterable)
10 GET_ITER
>> 11 FOR_ITER 30 (to 44)
14 STORE_FAST 3 (i)
17 LOAD_FAST 3 (i)
20 JUMP_IF_FALSE 17 (to 40)
23 POP_TOP
24 LOAD_FAST 2 (_[1])
27 LOAD_FAST 0 (func)
30 LOAD_FAST 3 (i)
33 CALL_FUNCTION 1
36 LIST_APPEND
37 JUMP_ABSOLUTE 11
>> 40 POP_TOP
41 JUMP_ABSOLUTE 11
>> 44 DELETE_FAST 2 (_[1])
47 RETURN_VALUE
>>>
相同的代码，外加一些代码以避免LIST_APPEND.
The same code, plus some code to avoid the LIST_APPEND.
在Python 3.X中，您需要更深入地研究:
>>> import dis
>>> def comprehension(f, iterable): return [f(i) for i in iterable]
...
>>> dis.dis(comprehension)
1 0 LOAD_CLOSURE 0 (f)
3 BUILD_TUPLE 1
6 LOAD_CONST 1 (<code object <listcomp> at 0x00C4B8D
8, file "<stdin>", line 1>)
9 MAKE_CLOSURE 0
12 LOAD_FAST 1 (iterable)
15 GET_ITER
16 CALL_FUNCTION 1
19 RETURN_VALUE
>>> dis.dis(comprehension.__code__.co_consts[1])
1 0 BUILD_LIST 0
3 LOAD_FAST 0 (.0)
>> 6 FOR_ITER 18 (to 27)
9 STORE_FAST 1 (i)
12 LOAD_DEREF 0 (f)
15 LOAD_FAST 1 (i)
18 CALL_FUNCTION 1
21 LIST_APPEND 2
24 JUMP_ABSOLUTE 6
>> 27 RETURN_VALUE
>>>
这是相同的旧方法:首先构建一个空列表，然后遍历可迭代对象，并根据需要追加到列表中.我在这里看不到任何预分配.
您正在考虑的优化用于单个操作码内，例如如果iterable可以准确报告其长度，则list.extend(iterable)的实现可以预分配. list.append(object)被赋予单个对象，而不是可迭代的对象.
The optimisation that you are thinking about is used inside a single opcode e.g. the implementation of list.extend(iterable) can preallocate if iterable can accurately report its length. list.append(object) is given a single object, not an iterable.
这篇关于Python 3:为range(N)中的i创建[func(i)]列表理解的最有效方法的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
