Spyder不会释放Matplotlib图的内存 [英] Spyder does not release memory for matplotlib plots

问题描述

我将Spyder 3.3.6与Pyhton 3.7.3搭配使用IPyhton 7.7.0 cosole.

出于不同的原因，Spyder一直是我选择的IDE，但是现在我决定完全放弃它，只坚持使用命令提示符.

原因是我在使用由Spyder引起的matplotlib时遇到了严重的内存泄漏.

内存线性增加，直到我的所有128 GB内存被消耗到我的Ubuntu计算机完全停止的扩展，我不得不对其进行硬重置.过去，我在使用matplotlib和找到的解决方案时遇到过此问题，例如使用

plt.close('all')或 plt.close(fig)或 gc.collect()

在某种程度上有所帮助，但这次没有任何效果(我应该补充一点，我正在使用MNE库绘制一长串EEG原始文件，因此，我无法尝试解决此类matplotlib内存问题的所有可能解决方案)./p>

但是当我在命令提示符下同时使用python和ipython运行相同的代码时，很简单

  plt.close(fig) 

解决了该问题，并且在整个运行过程中，内存保持在几GB的数量级.因此，唯一合乎逻辑的结论是Spyder在某种程度上与内存管理混为一谈.

因此，我写了这个问题，以防万一有人知道该问题的解决方案，或者如果不知道，那么使用Spyder并可能浪费大量时间试图找到python解决方案的其他人会知道问题出在Spyder，而不是python.

解决方案

对我来说，使用 gc.collect 解决了该问题.我正在使用:Spyder 3.3.6，Python 3.7.6，Ipython 7.11.1.

最小的工作/崩溃示例

下面的代码不会崩溃，但是可以通过注释/取消注释指示的两行来进行修改以导致内存泄漏.

 将numpy导入为np导入matplotlib.pylab作为plt导入操作系统导入psutil导入gc进程= psutil.Process(os.getpid())def str_mem_usage():mu = process.memory_info().rss/1024 ** 2返回内存使用量:{:. 2f} MB" .format(mu)arrs = []对于范围在(10)中的ii:print('it'，ii，'\ t'，str_mem_usage())n = 10000arr = np.random.rand(n ** 2).reshape(n，n)#arrs + = [arr]#激活显然导致内存泄漏plt.ioff()无花果，ax = plt.subplots(1，1)ax.imshow(arr)#plt.savefig('tmp.pdf')#与内存泄漏无关plt.close(图)plt.ion()gc.collect()#停用会导致内存泄漏 

此行为与在Spyder中执行或从Anaconda发行版调用Python或iPython时是相同的.

输出(无内存泄漏)

  it 0内存使用情况:147.35 MBit 1内存使用量:1682.64 MBit 2内存使用:1682.98 MBit 3内存使用:1682.99 MBit 4内存使用:1682.99 MBit 5内存使用:1682.99 MBit 6内存使用:1678.63 MBit 7内存使用:1644.47 MBit 8内存使用:1633.58 MBit 9内存使用:1633.56 MB 

输出(内存泄漏)

  it 0内存使用情况:108.89 MBit 1内存使用:1635.26 MBit 2内存使用量:2393.71 MB3内存使用量:3156.51 MBit 4内存使用情况:3919.34 MBit 5内存使用情况:4681.07 MBit 6内存使用:5428.71 MB...MemoryError:无法分配763.MiB用于形状为(100000000，)和数据类型为float64的数组 

也许这个答案可以帮助某些人重现/解决Matplotlib内存泄漏问题.这可能是我第五次遇到它.

I use Spyder 3.3.6, with Pyhton 3.7.3 over IPyhton 7.7.0 cosole.

For different reasons, Spyder has always been my IDE of choice, but now I have decided to drop it altogether and stick to command prompt only.

The reason is that I have seen a serious memory leak while using matplotlib which is caused by Spyder.

Memory increases linearly until all my 128 GB of memory is consumed to the extend that my Ubuntu machine halted completely and I had to hard reset it. I have had this issue in the past with matplotlib and the solution I found, things like using

plt.close('all') or plt.close(fig) or gc.collect()

helped to some extent, but this time none worked (I should add that I am using MNE library to plot a long list of EEG raw files, hence I cannot try every possible solution that exists for such matplotlib memory issues).

But when I ran the same code using both python and ipython in command prompt, a simple

plt.close(fig) 

solved the issue and the memory stays in the order of a few GB during the whole run. Hence, the only logical conclusion is that Spyder somehow messes with the memory management.

So, I wrote this question in case somebody knows a solution to this issue or if not, other people who use Spyder and probably have wasted hours trying to find a python solution would know the issue is in Spyder, not python.

解决方案

For me, using gc.collect resolved the issue. I am using: Spyder 3.3.6, Python 3.7.6, Ipython 7.11.1.

Minimal working/crashing example

The code below does not crash but can be modified to cause a memory leak by commenting/uncommenting the two lines indicated.

import numpy as np
import matplotlib.pylab as plt
import os
import psutil
import gc
process = psutil.Process(os.getpid())
def str_mem_usage():
    mu = process.memory_info().rss / 1024**2
    return 'Memory usage: {:.2f} MB'.format(mu)
    
arrs = []
for ii in range(10):
    print('it', ii, '\t', str_mem_usage())
    n = 10000
    arr = np.random.rand(n**2).reshape(n, n)
    #arrs += [arr]  # activate to cause memory leak, obviously
    
    plt.ioff()
    fig, ax = plt.subplots(1, 1)
    ax.imshow(arr)
    #plt.savefig('tmp.pdf')  # irrelevant for memory leak
    plt.close(fig)
    plt.ion()
    
    gc.collect()  # deactivate to cause memory leak

This behavior is the same for execution in Spyder or calling Python or iPython from the Anaconda distribution.

Output (no memory leak)

it 0     Memory usage: 147.35 MB
it 1     Memory usage: 1682.64 MB
it 2     Memory usage: 1682.98 MB
it 3     Memory usage: 1682.99 MB
it 4     Memory usage: 1682.99 MB
it 5     Memory usage: 1682.99 MB
it 6     Memory usage: 1678.63 MB
it 7     Memory usage: 1644.47 MB
it 8     Memory usage: 1633.58 MB
it 9     Memory usage: 1633.56 MB

Output (memory leak)

it 0     Memory usage: 108.89 MB
it 1     Memory usage: 1635.26 MB
it 2     Memory usage: 2393.71 MB
it 3     Memory usage: 3156.51 MB
it 4     Memory usage: 3919.34 MB
it 5     Memory usage: 4681.07 MB
it 6     Memory usage: 5428.71 MB
...
MemoryError: Unable to allocate 763. MiB for an array with shape (100000000,) and data type float64

Maybe this answer helps some people to reproduce / solve this Matplotlib memory leak issue. This is probably the 5th time I've come across it.

这篇关于Spyder不会释放Matplotlib图的内存的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！