使用matplotlib发生内存泄漏 [英] memory leaks using matplotlib
问题描述
这并非旨在作为错误报告-即使这些泄漏可能是mpl错误的结果，也请解释该问题，并寻求解决方法.
This is not intended as a bug report--even if these leaks may be a result of mpl bugs, please interpret the question ask asking for a way around them.
问题很简单:绘制大量数据(使用plot()或scatter())，清除/释放所有内容，进行垃圾回收，但仍然不是几乎所有的内存都被释放.
The problem is simple: plot a large chunk of data (using plot() or scatter()), clear/release everything, garbage collect, but still not nearly all the memory is released.
Line # Mem usage Increment Line Contents
================================================
391 122.312 MiB 0.000 MiB @profile
392 def plot_network_scatterplot(t_sim_stop, spikes_mat, n_cells_per_area, n_cells, basedir_output, condition_idx):
393
394 # make network scatterplot
395 122.312 MiB 0.000 MiB w, h = plt.figaspect(.1/(t_sim_stop/1E3))
396 122.324 MiB 0.012 MiB fig = mpl.figure.Figure(figsize=(10*w, 10*h))
397 122.328 MiB 0.004 MiB canvas = FigureCanvas(fig)
398 122.879 MiB 0.551 MiB ax = fig.add_axes([.01, .1, .98, .8])
399 134.879 MiB 12.000 MiB edgecolor_vec = np.array([(1., 0., 0.), (0., 0., 1.)])[1-((spikes_mat[:,3]+1)/2).astype(np.int)]
400 '''pathcoll = ax.scatter(spikes_mat[:,1],
401 spikes_mat[:,0] + n_cells_per_area * (spikes_mat[:,2]-1),
402 s=.5,
403 c=spikes_mat[:,3],
404 edgecolor=edgecolor_vec)'''
405 440.098 MiB 305.219 MiB pathcoll = ax.plot(np.random.rand(10000000), np.random.rand(10000000))
406 440.098 MiB 0.000 MiB ax.set_xlim([0., t_sim_stop])
407 440.098 MiB 0.000 MiB ax.set_ylim([1, n_cells])
408 440.098 MiB 0.000 MiB plt.xlabel('Time [ms]')
409 440.098 MiB 0.000 MiB plt.ylabel('Cell ID')
410 440.098 MiB 0.000 MiB plt.suptitle('Network activity scatterplot')
411 #plt.savefig(os.path.join(basedir_output, 'network_scatterplot-[cond=' + str(condition_idx) + '].png'))
412 931.898 MiB 491.801 MiB canvas.print_figure(os.path.join(basedir_output, 'network_scatterplot-[cond=' + str(condition_idx) + '].png'))
413 #fig.canvas.close()
414 #pathcoll.set_offsets([])
415 #pathcoll.remove()
416 931.898 MiB 0.000 MiB ax.cla()
417 931.898 MiB 0.000 MiB ax.clear()
418 931.898 MiB 0.000 MiB fig.clf()
419 931.898 MiB 0.000 MiB fig.clear()
420 931.898 MiB 0.000 MiB plt.clf()
421 932.352 MiB 0.453 MiB plt.cla()
422 932.352 MiB 0.000 MiB plt.close(fig)
423 932.352 MiB 0.000 MiB plt.close()
424 932.352 MiB 0.000 MiB del fig
425 932.352 MiB 0.000 MiB del ax
426 932.352 MiB 0.000 MiB del pathcoll
427 932.352 MiB 0.000 MiB del edgecolor_vec
428 932.352 MiB 0.000 MiB del canvas
429 505.094 MiB -427.258 MiB gc.collect()
430 505.094 MiB 0.000 MiB plt.close('all')
431 505.094 MiB 0.000 MiB gc.collect()
我尝试了所有清除/发布的许多组合和不同顺序，但无济于事.我试过不使用显式的无花果/画布创建，而只是使用mpl.pyplot，具有相同的结果.
I have tried many combinations and different orders of all the clear/release to no avail. I've tried not using an explicit fig/canvas creation but just using mpl.pyplot, with the same results.
有没有任何方式来释放此内存，并与我进来的122.312一起出去?
Is there any way to free this memory, and go out with the 122.312 that I came in?
干杯！
推荐答案
通常来说，将内存还给操作系统"的过程非常困难. (当然，直到进程终止并且操作系统取回所有内存为止) 因为(在大多数实现中)malloc返回的内容是用大块划分出来的 为了提高效率，但是如果其中任何一部分 仍在使用中."因此，您认为是内存泄漏可能只是副作用 这.如果是这样，fork可以解决问题.
It's very hard, in general, for a process to "give memory back to the OS" (until the process terminates and the OS gets back all the memory, of course) because (in most implementation) what malloc returns is carved out of big blocks for efficiency, but the whole block can't be given back if any part of it is still in use." So what you think is a memory leak may just be a side effect of this. If so, fork can solve the problem.
此外，
唯一真正可靠的方法来确保 暂时使用内存不会在完成后将所有资源返回给系统， 是要在子进程中进行这种使用，该进程需要大量内存 然后终止."
The only really reliable way to ensure that a large but temporary use of memory DOES return all resources to the system when it's done, is to have that use happen in a subprocess, which does the memory-hungry work then terminates."
因此，您不必尝试清除图形和轴，删除引用和垃圾收集(所有这些都将不起作用)，而可以使用multiprocessing在单独的过程中运行plot_network_scatterplot:
Therefore, you instead of trying to clear the figure and axes, delete references and garbage collecting (all of which will not work), you can instead use multiprocessing to run plot_network_scatterplot in a separate process:
import multiprocessing as mp
def plot_network_scatterplot(
t_sim_stop, spikes_mat, n_cells_per_area, n_cells, basedir_output,
condition_idx):
# make network scatterplot
w, h = plt.figaspect(.1/(t_sim_stop/1E3))
fig = mpl.figure.Figure(figsize=(10*w, 10*h))
canvas = FigureCanvas(fig)
ax = fig.add_axes([.01, .1, .98, .8])
edgecolor_vec = np.array([(1., 0., 0.), (0., 0., 1.)])[1-((spikes_mat[:,3]+1)/2).astype(np.int)]
'''pathcoll = ax.scatter(spikes_mat[:,1],
spikes_mat[:,0] + n_cells_per_area * (spikes_mat[:,2]-1),
s=.5,
c=spikes_mat[:,3],
edgecolor=edgecolor_vec)'''
pathcoll = ax.plot(np.random.rand(10000000), np.random.rand(10000000))
ax.set_xlim([0., t_sim_stop])
ax.set_ylim([1, n_cells])
plt.xlabel('Time [ms]')
plt.ylabel('Cell ID')
plt.suptitle('Network activity scatterplot')
canvas.print_figure(os.path.join(basedir_output, 'network_scatterplot-[cond=' + str(condition_idx) + '].png'))
def spawn(func, *args):
proc = mp.Process(target=func, args=args)
proc.start()
# wait until proc terminates.
proc.join()
if __name__ == '__main__':
spawn(plot_network_scatterplot, t_sim_stop, spikes_mat, n_cells_per_area,
n_cells, basedir_output, condition_idx)
这篇关于使用matplotlib发生内存泄漏的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
