通过返回迭代器而不是列表来保护Py3k内存 [英] Py3k memory conservation by returning iterators rather than lists

问题描述

用于在Python 2.x中返回列表的许多方法现在似乎在Py3k中返回迭代器

Many methods that used to return lists in Python 2.x now seem to return iterators in Py3k

迭代器是否也是生成器表达式？懒惰评估？

Are iterators also generator expressions? Lazy evaluation?

因此，这样python的内存占用量将大幅减少。不是吗？

Thus, with this the memory footprint of python is going to reduce drastically. Isn't it?

使用内置脚本从2到3转换的程序怎么样？

What about for the programs converted from 2to3 using the builtin script?

内置工具显式地将所有返回的迭代器转换为列表，以实现兼容性？如果是这样，那么Py3k的较低内存占用优势在转换后的程序中并不明显。是吗？

Does the builtin tool explicitly convert all the returned iterators into lists, for compatibility? If so then the lower memory footprint benefit of Py3k is not really apparent in the converted programs. Is it?

推荐答案

其中许多不完全是迭代器，而是特殊的视图对象。例如，range（）现在返回类似于旧xrange对象的东西 - 它仍然可以被索引，但是懒惰地根据需要构造整数。

Many of them are not exactly iterators, but special view objects. For instance range() now returns something similar to the old xrange object - it can still be indexed, but lazily constructs the integers as needed.

类似于dict.keys（）给出一个dict_keys对象，在dict上实现一个视图，而不是创建一个带有密钥副本的新列表。

Similarly dict.keys() gives a dict_keys object implementing a view on the dict, rather than creating a new list with a copy of the keys.

这对内存占用的影响可能取决于程序。当然除非你真的需要列表，否则更多的重点是使用迭代器，而使用列表通常是python2中的默认情况。这将导致平均程序可能更具内存效率。有很多节省的情况可能已经作为python2程序中的迭代器实现了，但是，因为真正大的内存使用会很突出，并且更有可能已经解决。 （例如，文件迭代器已经比旧的 file.readlines（）方法具有更高的内存效率）

How this affects memory footprints probably depends on the program. Certainly there's more of an emphasis towards using iterators unless you really need lists, whereas using lists was generally the default case in python2. That will cause the average program to probably be more memory efficient. Cases where there are really big savings are probably going to already be implemented as iterators in python2 programs however, as really large memory usage will stand out, and is more likely to be already addressed. (eg. the file iterator is already much more memory efficient than the older file.readlines() method)

转换由2to3工具完成，通常会将range（）之类的东西转换为迭代器，它可以安全地确定不需要真实列表，因此代码如下：

Converting is done by the 2to3 tool, and will generally convert things like range() to iterators where it can safely determine a real list isn't needed, so code like:

for x in range(10): print x

将切换到新的range（）对象，不再创建列表，因此将获得减少的内存优势，但代码如下：

will switch to the new range() object, no longer creating a list, and so will obtain the reduced memory benefit, but code like:

x = range(20)

将转换为：

x = list(range(20))

因为转换器无法知道代码是否需要x中的真实列表对象。

as the converter can't know if the code expects a real list object in x.

这篇关于通过返回迭代器而不是列表来保护Py3k内存的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！