删除字典中的项目后，Python回收内存 [英] Python reclaiming memory after deleting items in a dictionary

问题描述

我在Python中有一个比较大的字典，希望不仅能够从其中删除项目，而且实际上能够从程序中的这些删除中回收内存.我遇到了一个问题，尽管我从字典中删除了项目，甚至手动运行了垃圾收集器，但Python似乎并没有释放内存本身.

I have a relatively large dictionary in Python and would like to be able to not only delete items from it, but actually reclaim the memory back from these deletions in my program. I am running across a problem whereby although I delete items from the dictionary and even run the garbage collector manually, Python does not appear to be freeing the memory itself.

一个简单的例子:

>>> tupdict = {}
# consumes around 2 GB of memory
>>> for i in xrange(12500000):
...   tupdict[i] = (i,i)
... 
# delete over half the entries, no drop in consumed memory
>>> for i in xrange(7500000):
...   del tupdict[i]
... 
>>> import gc
# manually garbage collect, still no drop in consumed memory after this
>>> gc.collect()
0
>>> 

我想这是怎么回事，尽管删除了条目并运行了垃圾回收器，但是Python并没有继续调整字典的大小.我的问题是，有什么简单的方法可以解决这个问题，还是我可能需要对我的程序编写方式进行更认真的考虑?

I imagine what is happening is that although the entries are deleted and garbage collector run, Python does not go ahead and resize the dictionary. My question is, is there any simple way around this, or am I likely to require a more serious rethink about how I write my program?

推荐答案

Python是否将此内存返回给底层操作系统有很多因素，这可能是您试图确定是否释放了内存的方式. CPython有一个池化的分配器系统，该系统倾向于保留释放的内存，以便可以有效地重用它(但是从OS的角度来看，这些后续分配不会增加您的内存占用)，这可能就是您所需要的.重新看到.

A lot of factors go into whether Python returns this memory to the underlying OS or not, which is probably how you're trying to tell if memory is being freed. CPython has a pooled allocator system that tends to hold on to freed memory so that it can be reused in an efficient manner (but these subsequent allocations won't increase your memory footprint from the perspective of the OS), which might be what you're seeing.

此外，在某些unix平台上，进程不会在应用程序关闭(或发生其他一些重要事件)之前将释放的内存释放回OS.即使您处于整个池已被释放的情况(因此Python可能会决定将其释放()而不是将其保持为将来的对象打开状态)，OS仍不会释放该内存以供其他进程使用(但可用于原始流程中的进一步重新分配).通常，这对于减少内存碎片很有好处，并且没有太多的缺点，因为未使用的进程内存将被分页到磁盘上. Windows 确实将进程内存释放回操作系统，以供任何新分配使用(然后您可以在任务管理器中看到)，因此在Windows上尝试此操作可能会出现给您不同的结果.

Also, on some unix platforms processes don't release freed memory back to the OS until the application closes (or some other significant event occurs). Even if you are in a situation where an entire pool has been freed (and thus Python might decide to free() it rather than holding it open for future objects), the OS still won't release this memory to be used by other processes (but can be used for further reallocation within the original process). In general this is good for reducing memory fragmentation and doesn't have too much of a downside, as the unused process memory will get paged out to disk. Windows does release process memory back to the OS for use by any new allocation (which you can then see in the Task Manager), so trying this on Windows will likely appear to give you a different result.

最后，操作系统的工作范围是如何管理释放的进程内存，并且使用了各种方案(有优点也有缺点)，因此仅查看所选的系统信息工具并不一定会告诉您整个事实.

In the end, how to manage deallocated process memory is the purview of the operating system, and there are various schemes (with upsides and downsides) used such that just looking in your system information tool of choice won't necessarily tell you the whole truth.

这篇关于删除字典中的项目后，Python回收内存的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！