python中生成器对象的大小 [英] size of generator object in python

问题描述

对于以下代码:

import sys
x=(i for i in range(1,11))
print x


print 'Before starting iterating generator size is' ,sys.getsizeof(x)

print 'For first time'
for i in x:
    print i

print 'For second time , does not print anything'    
for i in x:
    print i # does not print anything

print 'After iterating generator size is' ,sys.getsizeof(x)

输出为:

<generator object <genexpr> at 0x014C1A80>
Before starting iterating generator size is 40
For first time
1
2
3
4
5
6
7
8
9
10
For second time
After iterating generator size is 40

最初生成器对象的大小是40，当我完成迭代时，它仍然是40.但是第二个循环中没有引用任何元素.

The size of generator object at first is 40, when I finished with iterating it is still 40. But no element is referenced from the second loop.

为什么生成器对象在创建时和完成迭代后会占用相同的内存?

Why does the generator object take the same memory when it was created and as it does when finished iterating over it?

推荐答案

生成器在内存中占用的空间为只是簿记信息.在其中保留对框架对象的引用(对正在运行的Python代码的管理，例如本地语言)，无论它是否正在运行，都保留对代码对象的引用.没什么了

The space a generator takes in memory is just bookkeeping info. In it a reference to the frame object is kept (administration for the running Python code, such as locals), wether or not it is running right now, and a reference to the code object are kept. Nothing more:

>>> x=(i for i in range(1,11))
>>> dir(x)
['__class__', '__delattr__', '__doc__', '__format__', '__getattribute__', '__hash__', '__init__', '__iter__', '__name__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'close', 'gi_code', 'gi_frame', 'gi_running', 'next', 'send', 'throw']
>>> x.gi_frame
<frame object at 0x1053b4ad0>
>>> x.gi_running
0
>>> x.gi_code
<code object <genexpr> at 0x1051af5b0, file "<stdin>", line 1>

只有3个引用，再加上通常的Python对象类型信息(考虑引用计数)和弱引用列表；因此大约有4个指针，一个整数和一个结构，它们在您的系统上占用40个字节(在我的系统上，它是64位OS X，它是80个字节). sys.getsizeof()报告使用C语言实现的 just 结构的大小，并且不会在指针上递归.

That's just 3 references, plus the usual Python object type info (think reference counting) and a weak-references list; so that's about 4 pointers, an integer and a struct, which on your system take 40 bytes (on my system, 64-bit OS X, it is 80 bytes). sys.getsizeof() reports on the size of just that structure as implemented in C, and it doesn't recurse over pointers.

这样，当您通过生成器运行时，该内存量将不会更改.所引用的帧可能会改变所使用的内存量(如果生成器表达式将大型对象指向一端或另一端)，但是生成器对象上的sys.getsizeof()结果将不会导致这种情况；而是查看框架本地人:

As such, that amount of memory won't change when you have run through the generator. The referenced frame may change in how much memory is used (if the generator expression references large objects towards one end or the other) but you won't see that with the result of sys.getsizeof() on the generator object; look at the frame locals instead:

>>> next(x)
1
>>> x.gi_frame.f_locals
{'i': 1, '.0': <listiterator object at 0x105339dd0>}

.0对象是生成器在for循环中使用的range()迭代器，i是for循环的目标. listiterator是另一个可迭代对象，它具有对生成的列表range()的私有引用以及位置计数器，因此可以在每次请求时产生下一个元素.

The .0 object is the range() iterator that the generator is using in the for loop, i is the for loop target. The listiterator is another iterable object that has a private reference to the list range() produced as well as a position counter so it can yield the next element each time you ask it to.

您无法查询生成器的元素大小；无论如何，它们都会根据需要生成元素，您不能先验地知道"它们将要生成多少，也无法知道它们在运行后已经生成了多少. sys.getsizeof()当然不会告诉您；无论如何，它都是一种测量内存占用量的工具，如果您想知道 total 占用空间，则必须递归地测量所有引用的对象.

You cannot query for an element size of a generator; they produce elements as needed anyway, you cannot a-priori 'know' how much they'll produce, nor know how much they have produced after running. sys.getsizeof() certainly won't tell you; it is a tool to measure memory footprint anyway, and you'd have to recursively measure all referenced objects if you want to know the total footprint.

您可以看到生成器已从框架完成运行；完成后，将清除:

You can see that the generator has completed its run from the frame; it is cleared once it is done:

>>> x.gi_frame
<frame object at 0x1053b4ad0>
>>> list(x)
[2, 3, 4, 5, 6, 7, 8, 9, 10]
>>> x.gi_frame is None
True

因此，最后，用于生成器的内存驻留在框架中的结构中(本地，可能还有全局，这些命名空间中的每个对象可能再次引用其他对象)，并且在生成器完成时，将清除框架并且将生成器.gi_frame指针更改为指向None单例，如果引用计数降至0，则需要清除该帧.

So in the end, the memory used for the generator resides in structures in the frame (locals, and possibly globals, with each object in those namespaces possibly referencing other objects again), and when the generator is done the frame is cleared and the generator .gi_frame pointer is altered to point to the None singleton, leaving the frame to be cleared if the reference count has dropped to 0.

所有这些仅适用于 generators ，而不适用于一般的可迭代对象.生成器是Python代码，因此可以对此进行深入的反思.

All this only applies to generators, not to iterables in general; generators are Python code and thus can be introspected this deeply.

这篇关于python中生成器对象的大小的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！