Python 切片分配内存使用 [英] Python Slice Assignment Memory Usage

问题描述

我在 Stack Overflow 上的评论中读到，在更改列表时进行切片分配的内存效率更高.例如，

I read in a comment here on Stack Overflow that it is more memory efficient to do slice assignment when changing lists. For example,

a[:] = [i + 6 for i in a]

应该比

a = [i + 6 for i in a]

因为前者替换现有列表中的元素，而后者创建一个新列表并将 a 重新绑定到该新列表，将旧的 a 留在内存中，直到它可以垃圾回收.在速度方面对两者进行基准测试，后者稍快一些:

because the former replaces elements in the existing list, while the latter creates a new list and rebinds a to that new list, leaving the old a in memory until it can be garbage collected. Benchmarking the two for speed, the latter is slightly quicker:

$ python -mtimeit -s 'a = [1, 2, 3]' 'a[:] = [i + 6 for i in a]'
1000000 loops, best of 3: 1.53 usec per loop
$ python -mtimeit -s 'a = [1, 2, 3]' 'a = [i + 6 for i in a]'
1000000 loops, best of 3: 1.37 usec per loop

这正是我所期望的，因为重新绑定变量应该比替换列表中的元素更快.但是，我找不到任何支持内存使用声明的官方文档，我不确定如何对其进行基准测试.

That is what I'd expect, as rebinding a variable should be faster than replacing elements in a list. However, I can't find any official documentation which supports the memory usage claim, and I'm not sure how to benchmark that.

从表面上看，内存使用声明对我来说很有意义.然而，再考虑一下，我希望在前一种方法中，解释器会从列表推导中创建一个新列表，然后然后将该列表中的值复制到 a，让匿名列表四处漂浮，直到它被垃圾收集.如果是这种情况，那么前一种方法将使用相同数量的内存，但速度也较慢.

On the face of it, the memory usage claim makes sense to me. However, giving it some more thought, I would expect that in the former method, the interpreter would create a new list from the list comprehension and then copy the values from that list to a, leaving the anonymous list in floating around until it is garbage collected. If that's the case, then the former method would use the same amount of memory while also being slower.

任何人都可以明确地(通过基准测试或官方文档)这两种方法中哪一种内存效率更高/哪一种是首选方法?

Can anyone show definitively (with a benchmark or official documentation) which of the two methods is more memory efficient/which is the preferred method?

提前致谢.

推荐答案

行

a[:] = [i + 6 for i in a]

不会保存任何内存.正如语言文档中所述，Python 确实首先评估了右侧:

would not save any memory. Python does evaluate the right hand side first, as stated in the language documentation:

赋值语句计算表达式列表(记住这可以是单个表达式或逗号分隔的列表，后者产生一个元组)并将单个结果对象从左到右分配给每个目标列表.

An assignment statement evaluates the expression list (remember that this can be a single expression or a comma-separated list, the latter yielding a tuple) and assigns the single resulting object to each of the target lists, from left to right.

在手头的情况下，单个结果对象将是一个新列表，目标列表中的单个目标将是 a[:].

In the case at hand, the single resulting object would be a new list, and the single target in the target list would be a[:].

我们可以用生成器表达式替换列表推导式:

We could replace the list comprehension by a generator expression:

a[:] = (i + 6 for i in a)

现在，右侧的计算结果是生成器而不是列表.基准测试表明，这仍然比naive慢

Now, the right hand side evaluates to a generator instead of a list. Benchmarking shows that this is still slower than the naive

a = [i + 6 for i in a]

那么生成器表达式真的节省了内存吗?乍一看，您可能认为确实如此.但是深入研究该函数的源代码list_ass_slice() 表明它没有.线

So does the generator expression actually save any memory? At first glance, you might think it does. But delving in to the source code of the function list_ass_slice() shows that it does not. The line

v_as_SF = PySequence_Fast(v, "can only assign an iterable");

使用 PySequence_Fast() 来转换可迭代对象(在本例中生成器)首先转换为元组，然后将其复制到旧列表中.元组使用与列表相同的内存量，因此在这种情况下使用生成器表达式与使用列表推导式基本相同.在最后一次复制期间，原始列表的项目被重用.

uses PySequence_Fast() to convert the iterable (in this case the generator) into a tuple first, which is then copied into the old list. A tuple uses the same amount of memory as a list, so using a generator expression is basically the same as using a list comprehension in this case. During the last copy, the items of the original list are reused.

道德似乎是最简单的方法在任何方面都是最好的.

The moral seems to be that the simplest approach is the best one in any regard.

这篇关于Python 切片分配内存使用的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！