Spyder无法为Matplotlib图重新分配内存 [英] Spyder does not realease memory for matplotlib plots

问题描述

我将Spyder 3.3.6与Pyhton 3.7.3搭配使用IPyhton 7.7.0 cosole.

出于不同的原因，Spyder一直是我选择的IDE，但是现在我决定完全放弃它，只坚持使用命令提示符.

原因是我在使用由Spyder引起的matplotlib时遇到了严重的内存泄漏.

内存线性增加，直到我的所有128 GB内存被消耗到我的Ubuntu计算机完全停止的扩展，我不得不对其进行硬重置.过去，我在使用matplotlib和找到的解决方案时遇到过此问题，例如使用

plt.close('all')或plt.close(fig)或gc.collect()

在某种程度上有所帮助，但是这次没有任何效果(我应该补充一点，我正在使用MNE库绘制一长串EEG原始文件，因此，我无法尝试解决此类matplotlib内存问题的所有可能解决方案). /p>

但是当我在命令提示符下同时使用python和ipython运行相同的代码时，很简单

 plt.close(fig) 
 

解决了该问题，并且在整个运行过程中，内存保持在几GB的数量级.因此，唯一合乎逻辑的结论是Spyder在某种程度上与内存管理混乱.

因此，我写了这个问题，以防万一有人知道该问题的解决方案，或者如果不知道，那么使用Spyder并可能浪费大量时间试图找到python解决方案的其他人会知道问题出在Spyder，而不是python.

解决方案

对我来说，使用gc.collect解决了该问题. 我正在使用:Spyder 3.3.6，Python 3.7.6，Ipython 7.11.1.

最小的工作/崩溃示例

import numpy as np
import matplotlib.pylab as plt
import os
import psutil
import gc
process = psutil.Process(os.getpid())
def str_mem_usage():
    mu = process.memory_info().rss / 1024**2
    return 'Memory usage: {:.2f} MB'.format(mu)

arrs = []
for ii in range(10):
    print('it', ii, '\t', str_mem_usage())
    n = 10000
    arr = np.random.rand(n**2).reshape(n, n)
    #arrs += [arr]  # activate for memory leak, obviously

    plt.ioff()
    fig, ax = plt.subplots(1, 1)
    ax.imshow(arr)
    #plt.savefig('tmp.pdf')  # irrelevant for memory leak
    plt.close(fig)
    plt.ion()

    gc.collect()  # deactivate for memory leak

此行为与在spyder中执行或从anaconda发行版调用python或ipython相同.

输出(无内存泄漏)

it 0     Memory usage: 147.35 MB
it 1     Memory usage: 1682.64 MB
it 2     Memory usage: 1682.98 MB
it 3     Memory usage: 1682.99 MB
it 4     Memory usage: 1682.99 MB
it 5     Memory usage: 1682.99 MB
it 6     Memory usage: 1678.63 MB
it 7     Memory usage: 1644.47 MB
it 8     Memory usage: 1633.58 MB
it 9     Memory usage: 1633.56 MB

输出(内存泄漏)

it 0     Memory usage: 108.89 MB
it 1     Memory usage: 1635.26 MB
it 2     Memory usage: 2393.71 MB
it 3     Memory usage: 3156.51 MB
it 4     Memory usage: 3919.34 MB
it 5     Memory usage: 4681.07 MB
it 6     Memory usage: 5428.71 MB
...
MemoryError: Unable to allocate 763. MiB for an array with shape (100000000,) and data type float64

也许这个答案可以帮助某些人重现/解决此matplotlib内存泄漏问题.我大概是第五次了.

I use Spyder 3.3.6, with Pyhton 3.7.3 over IPyhton 7.7.0 cosole.

For different reasons, Spyder has always been my IDE of choice, but now I have decided to drop it altogether and stick to command prompt only.

The reason is that I have seen a serious memory leak while using matplotlib which is caused by Spyder.

Memory increases linearly until all my 128 GB of memory is consumed to the extend that my Ubuntu machine halted completely and I had to hard reset it. I have had this issue in the past with matplotlib and the solution I found, things like using

plt.close('all') or plt.close(fig) or gc.collect()

helped to some extent, but this time none worked (I should add that I am using MNE library to plot a long list of EEG raw files, hence I cannot try every possible solution that exists for such matplotlib memory issues).

But when I ran the same code using both python and ipython in command prompt, a simple

plt.close(fig) 

solved the issue and the memory stays in the order of a few GB during the whole run. Hence, the only logical conclusion is that Spyder somehow messes with the memory management.

So, I wrote this question in case somebody knows a solution to this issue or if not, other people who use Spyder and probably have wasted hours trying to find a python solution would know the issue is in Spyder, not python.

解决方案

For me, using gc.collect resolved the issue. I am using: Spyder 3.3.6, Python 3.7.6, Ipython 7.11.1.

Minimal working/crashing example

import numpy as np
import matplotlib.pylab as plt
import os
import psutil
import gc
process = psutil.Process(os.getpid())
def str_mem_usage():
    mu = process.memory_info().rss / 1024**2
    return 'Memory usage: {:.2f} MB'.format(mu)

arrs = []
for ii in range(10):
    print('it', ii, '\t', str_mem_usage())
    n = 10000
    arr = np.random.rand(n**2).reshape(n, n)
    #arrs += [arr]  # activate for memory leak, obviously

    plt.ioff()
    fig, ax = plt.subplots(1, 1)
    ax.imshow(arr)
    #plt.savefig('tmp.pdf')  # irrelevant for memory leak
    plt.close(fig)
    plt.ion()

    gc.collect()  # deactivate for memory leak

This behaviour is the same for execution in spyder or calling python or ipython from the anaconda distribution.

Output (no memory leak)

it 0     Memory usage: 147.35 MB
it 1     Memory usage: 1682.64 MB
it 2     Memory usage: 1682.98 MB
it 3     Memory usage: 1682.99 MB
it 4     Memory usage: 1682.99 MB
it 5     Memory usage: 1682.99 MB
it 6     Memory usage: 1678.63 MB
it 7     Memory usage: 1644.47 MB
it 8     Memory usage: 1633.58 MB
it 9     Memory usage: 1633.56 MB

Output (memory leak)

it 0     Memory usage: 108.89 MB
it 1     Memory usage: 1635.26 MB
it 2     Memory usage: 2393.71 MB
it 3     Memory usage: 3156.51 MB
it 4     Memory usage: 3919.34 MB
it 5     Memory usage: 4681.07 MB
it 6     Memory usage: 5428.71 MB
...
MemoryError: Unable to allocate 763. MiB for an array with shape (100000000,) and data type float64

Maybe this answer helps some people to reproduce / solve this matplotlib memory leak issue. This is probably the 5th time, I step over it.

这篇关于Spyder无法为Matplotlib图重新分配内存的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！