了解python内存分配并释放 [英] understanding python memory allocation and free up
问题描述
我最近遇到了关于Python内存分配的这篇文章
I recently came across this article about python memory allocation.
在此页面中，它描述了python的内存使用情况，并且其中有一个示例显示了整数列表的深层复制.我自己在Python 2.7上进行了基准测试
In this page it describes the memory usage of python and in there there is an example showing deepcopy of list of integers. I did the benchmark myself on Python 2.7
Line # Mem usage Increment Line Contents
================================================
4 28.051 MiB 0.000 MiB @profile
5 def function():
6 59.098 MiB 31.047 MiB x = list(range(1000000)) # allocate a big list
7 107.273 MiB 48.176 MiB y = copy.deepcopy(x)
8 99.641 MiB -7.633 MiB del x
9 99.641 MiB 0.000 MiB return y
所以直接删除x只会删除x，并且所有对整数的引用都将x删除对吗?
so delete x directly only removes x and all the references to integer to x right?
这样做也无济于事(那么del x和del x [:]有什么区别?):
Doing this could not help either (So what is the difference del x and del x[:]?):
Line # Mem usage Increment Line Contents
================================================
4 28.047 MiB 0.000 MiB @profile
5 def function():
6 59.094 MiB 31.047 MiB x = list(range(1000000)) # allocate a big list
7 107.270 MiB 48.176 MiB y = copy.deepcopy(x)
8 99.637 MiB -7.633 MiB del x[:]
9 99.637 MiB 0.000 MiB return y
与Deepcopy相比，如果我使用copy，则在删除后，似乎在新创建x时内存将恢复到以前的状态
And in contrast to deepcopy, if I use copy, after deletion seems the memory restores to previous state when x is newly created
Line # Mem usage Increment Line Contents
================================================
4 28.039 MiB 0.000 MiB @profile
5 def function():
6 59.090 MiB 31.051 MiB x = list(range(1000000)) # allocate a big list
7 66.895 MiB 7.805 MiB y = copy.copy(x)
8 59.262 MiB -7.633 MiB del x[:]
9 59.262 MiB 0.000 MiB return y
对于字典:
Line # Mem usage Increment Line Contents
================================================
4 28.051 MiB 0.000 MiB @profile
5 def function():
6 100.523 MiB 72.473 MiB x = dict((e, e) for e in xrange(1000000))
7 183.398 MiB 82.875 MiB y = copy.deepcopy(x)
8 135.395 MiB -48.004 MiB del x
9 135.395 MiB 0.000 MiB return y
对于列表列表(与整数列表相比，我认为del x或del x [:]仅删除堆上的那个巨大数组列表?):
And for list of lists (compare to list of integers, I assume that del x or del x[:] only removes that huge array list on heap?):
Line # Mem usage Increment Line Contents
================================================
4 28.043 MiB 0.000 MiB @profile
5 def function():
6 107.691 MiB 79.648 MiB x = [[] for _ in xrange(1000000)]
7 222.312 MiB 114.621 MiB y = copy.deepcopy(x)
8 214.680 MiB -7.633 MiB del x[:]
9 214.680 MiB 0.000 MiB return y
所以我想问:
- 那么，如果没有办法索回那些整数所占用的内存?整数也是对象吗?为什么内存根本不释放?不能声明整数?还是浮点数和字符串?对象引用也一样?
- 为什么有-7 MB的内存?是因为列表(以数组列表形式实现)从堆中释放了?
- 无论是列表还是字典，del x只能释放数据结构本身(我的意思是数组列表结构或dict结构)，但是可以将整数，对象引用标记为空闲，但不能返回系统?
在这个示例中，我如何或者是否有办法释放x中的所有下划线列表?
And how do I or if there is a way to free all the underlining lists in x in this example?
Line # Mem usage Increment Line Contents
================================================
4 28.047 MiB 0.000 MiB @profile
5 def function():
6 248.008 MiB 219.961 MiB x = [list(range(10)) for _ in xrange(1000000)]
7 502.195 MiB 254.188 MiB y = copy.deepcopy(x)
8 494.562 MiB -7.633 MiB del x[:]
9 494.562 MiB 0.000 MiB return y
推荐答案
TL; DR
del不会像C中那样释放变量，它只是说您不再需要它了.然后发生的是实现细节.
TL;DR
del does not free variables as in C, it simply says that you no longer need it. What then happens is an implementation detail.
所以这里发生的是 del 不会不是的空闲内存，它只是告诉python您已经完成了变量操作.具体来说:
So what is happening here is that del does not free memory, it simply tells python that you are done with the variable. Specifically:
7.5. del语句
7.5. The del statement
del_stmt :: ="del" target_list
del_stmt ::= "del" target_list
递归定义删除与定义分配的方式非常相似.这里没有提示详细信息，而是提供了一些提示.
Deletion is recursively defined very similar to the way assignment is defined. Rather than spelling it out in full details, here are some hints.
删除目标列表会从左到右递归删除每个目标.
Deletion of a target list recursively deletes each target, from left to right.
名称的删除会从本地或全局名称空间中删除该名称的绑定，具体取决于该名称是否出现在同一代码块的全局语句中.如果名称未绑定，则会引发NameError异常.
Deletion of a name removes the binding of that name from the local or global namespace, depending on whether the name occurs in a global statement in the same code block. If the name is unbound, a NameError exception will be raised.
删除属性引用，订阅和切片将传递给所涉及的主要对象；删除切片通常等同于分配正确类型的空切片(但即使是由切片的对象确定).
Deletion of attribute references, subscriptions and slicings is passed to the primary object involved; deletion of a slicing is in general equivalent to assignment of an empty slice of the right type (but even this is determined by the sliced object).
请注意，没有提到释放内存.相反，发生的事情是您告诉python它可以对该内存执行任何所需的操作".在这种情况下，您的python实现(我假设是CPython)存储内存，以供以后在内存缓存中使用.这样可以使python更快地运行，而无需稍后分配更多的内存.
Note that there is no mention of freeing memory. What instead happens is that you tell python that it can do "whatever it wants" with that memory. In this case your python implementation (which I assume is CPython) stores the memory for later use in a memory cache. This allows python to run faster by not needing to allocate as much memory later.
请考虑以下示例，在此示例中，我们del x然后再次创建y的副本.请注意，第二个副本期间分配的内存量小于第一个副本期间分配的内存量.这是因为内存已被重用.如果再次执行此操作，则会发现在第三次复制期间几乎没有分配任何内存，因为python只是在重复使用先前分配的内存:
Consider this example, where we del x and then create a copy of y again. Note that the amount of memory allocated during the second copy is smaller than during the first. This is because memory is re-used. If we do this again, we see that hardly any memory at all is allocated during the third copy, because python is simply re-using previously allocated memory:
Line # Mem usage Increment Line Contents
================================================
4 34.777 MiB 0.000 MiB @profile
5 def function():
6 37.504 MiB 2.727 MiB x = [list(range(10)) for _ in xrange(10000)]
7 40.773 MiB 3.270 MiB y = copy.deepcopy(x)
8 40.773 MiB 0.000 MiB del x
9 41.820 MiB 1.047 MiB y2 = copy.deepcopy(y)
10 41.820 MiB 0.000 MiB del y2
11 41.824 MiB 0.004 MiB y3 = copy.deepcopy(y)
12 41.824 MiB 0.000 MiB return y
来源
优秀的博客": http://www.evanjones.ca/memoryallocator/
http://effbot.org/pyfaq/why-doesnt-python-release-the-memory-when-i-delete-a-large-object.htm
这篇关于了解python内存分配并释放的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
