Python切片分配内存使用情况 [英] Python Slice Assignment Memory Usage

问题描述

我在此处对Stack Overflow的评论中读到，更改列表时进行切片分配具有更高的内存效率.例如，

I read in a comment here on Stack Overflow that it is more memory efficient to do slice assignment when changing lists. For example,

a[:] = [i + 6 for i in a]

应该比

a = [i + 6 for i in a]

因为前者替换了现有列表中的元素，而后者创建了一个新列表并将a重新绑定到该新列表，从而将旧的a保留在内存中，直到可以对其进行垃圾回收为止.对两者进行基准测试以提高速度，后者则要快一些:

because the former replaces elements in the existing list, while the latter creates a new list and rebinds a to that new list, leaving the old a in memory until it can be garbage collected. Benchmarking the two for speed, the latter is slightly quicker:

$ python -mtimeit -s 'a = [1, 2, 3]' 'a[:] = [i + 6 for i in a]'
1000000 loops, best of 3: 1.53 usec per loop
$ python -mtimeit -s 'a = [1, 2, 3]' 'a = [i + 6 for i in a]'
1000000 loops, best of 3: 1.37 usec per loop

这就是我所期望的，因为重新绑定变量应该比替换列表中的元素更快.但是，我找不到任何支持内存使用声明的官方文档，也不确定如何对其进行基准测试.

That is what I'd expect, as rebinding a variable should be faster than replacing elements in a list. However, I can't find any official documentation which supports the memory usage claim, and I'm not sure how to benchmark that.

从表面上看，内存使用声明对我来说很有意义.但是，请多加考虑，我希望在前一种方法中，解释器将从列表理解中创建一个新列表，然后 then 将该列表中的值复制到a，从而使匿名列表会一直浮动，直到被垃圾回收为止.如果是这种情况，那么前一种方法将使用相同数量的内存，同时速度也会变慢.

On the face of it, the memory usage claim makes sense to me. However, giving it some more thought, I would expect that in the former method, the interpreter would create a new list from the list comprehension and then copy the values from that list to a, leaving the anonymous list in floating around until it is garbage collected. If that's the case, then the former method would use the same amount of memory while also being slower.

任何人都可以(通过基准测试或官方文档)明确显示这两种方法中哪一种更有效利用内存/哪一种是首选方法?

Can anyone show definitively (with a benchmark or official documentation) which of the two methods is more memory efficient/which is the preferred method?

谢谢.

推荐答案

行

a[:] = [i + 6 for i in a]

将不保存任何内存.如语言文档所述，Python确实会首先评估右侧. :

would not save any memory. Python does evaluate the right hand side first, as stated in the language documentation:

赋值语句评估表达式列表(请记住，它可以是单个表达式或逗号分隔的列表，后者产生一个元组)，并将单个结果对象从左到右分配给每个目标列表.

An assignment statement evaluates the expression list (remember that this can be a single expression or a comma-separated list, the latter yielding a tuple) and assigns the single resulting object to each of the target lists, from left to right.

在当前情况下，单个结果对象将是一个新列表，而目标列表中的单个目标将是a[:].

In the case at hand, the single resulting object would be a new list, and the single target in the target list would be a[:].

我们可以用生成器表达式代替列表推导:

We could replace the list comprehension by a generator expression:

a[:] = (i + 6 for i in a)

现在，右侧将求值为生成器，而不是列表.基准测试表明，这仍然比幼稚的速度慢

Now, the right hand side evaluates to a generator instead of a list. Benchmarking shows that this is still slower than the naive

a = [i + 6 for i in a]

那么生成器表达式实际上可以节省任何内存吗?乍一看，您可能会认为确实如此.但是请仔细阅读函数源代码list_ass_slice() 表示没有.线

So does the generator expression actually save any memory? At first glance, you might think it does. But delving in to the source code of the function list_ass_slice() shows that it does not. The line

v_as_SF = PySequence_Fast(v, "can only assign an iterable");

使用 PySequence_Fast()转换可迭代对象(在这种情况下，生成器)先放入一个元组，然后再复制到旧列表中.元组使用与列表相同的内存量，因此在这种情况下，使用生成器表达式与使用列表推导基本上相同.在最后一次复制期间，原始列表中的项目被重复使用.

uses PySequence_Fast() to convert the iterable (in this case the generator) into a tuple first, which is then copied into the old list. A tuple uses the same amount of memory as a list, so using a generator expression is basically the same as using a list comprehension in this case. During the last copy, the items of the original list are reused.

在道德上似乎是最简单的方法在任何方面都是最好的方法.

The moral seems to be that the simplest approach is the best one in any regard.

这篇关于Python切片分配内存使用情况的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！