Python-解决内存泄漏 [英] Python - Working around memory leaks

问题描述

我有一个Python程序，它运行一系列实验，没有打算从一个测试存储到另一个测试的数据.我的代码包含一个我完全找不到的内存泄漏(我查看了其他线程关于内存泄漏).由于时间限制，我不得不放弃寻找泄漏的机会，但是如果我能够隔离每个实验，该程序可能会运行足够长的时间以产生所需的结果.

I have a Python program that runs a series of experiments, with no data intended to be stored from one test to another. My code contains a memory leak which I am completely unable to find (I've look at the other threads on memory leaks). Due to time constraints, I have had to give up on finding the leak, but if I were able to isolate each experiment, the program would probably run long enough to produce the results I need.

• 是否可以在单独的线程帮助中运行每个测试?
• 还有其他隔离泄漏影响的方法吗?

有关具体情况的详细信息

• 我的代码分为两部分:实验运行程序和实际实验代码.
• 尽管用于运行所有实验的代码与每个实验使用的代码之间没有共享全局变量，但某些类/函数却必须共享.
• 实验运行程序不仅是可以轻松放入shell脚本的简单for循环.它首先根据给定的配置参数决定需要运行的测试，然后运行测试，然后以特定方式输出数据.
• 我尝试手动调用垃圾收集器，以防万一问题只是没有运行垃圾收集，但这没有用

更新

Gnibbler的答案实际上使我发现我的ClosenessCalculation对象存储了所有在每次计算过程中使用的所有数据并没有被杀死.然后，我用它来手动删除一些似乎已解决内存问题的链接.

Gnibbler's answer has actually allowed me to find out that my ClosenessCalculation objects which store all of the data used during each calculation are not being killed off. I then used that to manually delete some links which seems to have fixed the memory issues.

推荐答案

您可以使用类似的方法来帮助跟踪内存泄漏

You can use something like this to help track down memory leaks

>>> from collections import defaultdict
>>> from gc import get_objects
>>> before = defaultdict(int)
>>> after = defaultdict(int)
>>> for i in get_objects():
...     before[type(i)] += 1 
... 

现在假设测试会泄漏一些内存

now suppose the tests leaks some memory

>>> leaked_things = [[x] for x in range(10)]
>>> for i in get_objects():
...     after[type(i)] += 1
... 
>>> print [(k, after[k] - before[k]) for k in after if after[k] - before[k]]
[(<type 'list'>, 11)]

11，因为我们泄漏了一个包含10个以上列表的列表

11 because we have leaked one list containing 10 more lists

这篇关于Python-解决内存泄漏的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！