Python 2.6 GC似乎可以清除对象，但不会释放内存 [英] Python 2.6 GC appears to cleanup objects, but memory is not released

问题描述

我有一个用python 2.6编写的程序，该程序创建了大量的短期实例(这是经典的生产者－消费者问题).我注意到，创建这些实例时，top和pmap报告的内存使用情况似乎有所增加，并且从不回落.我担心我正在使用的某些python模块可能会泄漏内存，因此我在代码中仔细隔离了该问题.然后，我以尽可能短的示例进行再现.我想到了这个:

I have a program written in python 2.6 that creates a large number of short lived instances (it is a classic producer-consumer problem). I noticed that the memory usage as reported by top and pmap seems to increase when these instances are created and never goes back down. I was concerned that some python module I was using might be leaking memory so I carefully isolated the problem in my code. I then proceeded to reproduce it in as short as example as possible. I came up with this:

class LeaksMemory(list):
    timesDelCalled = 0

    def __del__(self):
        LeaksMemory.timesDelCalled +=1


def leakSomeMemory():
    l = []
    for i in range(0,500000):
        ml = LeaksMemory()
        ml.append(float(i))
        ml.append(float(i*2))
        ml.append(float(i*3))
        l.append(ml)

import gc
import os


leakSomeMemory()

print("__del__ was called " + str(LeaksMemory.timesDelCalled) + " times")
print(str(gc.collect())  +" objects collected")
print("__del__ was called " + str(LeaksMemory.timesDelCalled) + " times")
print(str(os.getpid()) + " : check memory usage with pmap or top")

如果使用类似"python2.6 -i memoryleak.py"的命令运行该命令，它将暂停，您可以使用pmap -x PID来检查内存使用情况.我添加了 del 方法，以便可以验证是否正在发生GC.它不在我的实际程序中，并且似乎没有任何功能上的差异.每次调用LeakSomeMemory()都会增加此程序消耗的内存量.我担心我犯了一个简单的错误，引用被偶然保存，但无法识别.

If you run this with something like 'python2.6 -i memoryleak.py' it will halt and you can use pmap -x PID to check the memory usage. I added the del method so I could verify that GC was occuring. It is not there in my actual program and does not appear to make any functional difference. Each call to leakSomeMemory() increases the amount of memory consumed by this program. I fear I am making some simple error and that references are getting kept by accident, but cannot identify it.

推荐答案

Python会释放对象，但不会立即将内存释放回操作系统.取而代之的是，它将在相同的解释器中重复使用相同的段以用于将来的分配.

Python will release the objects, but it will not release the memory back to the operating system immediately. Instead, it will re-use the same segments for future allocations within the same interpreter.

这是有关此问题的博客文章:

Here's a blog post about the issue: http://effbot.org/pyfaq/why-doesnt-python-release-the-memory-when-i-delete-a-large-object.htm

更新:我自己使用Python 2.6.4进行了测试，但没有注意到内存使用量的持续增加. leakSomeMemory()的某些调用导致Python进程的内存占用增加，而某些调用又使它减少了.因此，这完全取决于分配器如何重新使用内存.

UPDATE: I tested this myself with Python 2.6.4 and didn't notice persistent increases in memory usage. Some invocations of leakSomeMemory() caused the memory footprint of the Python process to increase, and some made it decrease again. So it all depends on how the allocator is re-using the memory.

这篇关于Python 2.6 GC似乎可以清除对象，但不会释放内存的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！