清除numpy.memmap [英] Purging numpy.memmap

问题描述

给出一个用 mode ='r'创建的 numpy.memmap 对象(即只读)，有一种方法可以强制它清除所有已加载的页面物理RAM，而不删除对象本身?

Given a numpy.memmap object created with mode='r' (i.e. read-only), is there a way to force it to purge all loaded pages out of physical RAM, without deleting the object itself?

换句话说，我希望对 memmap 实例的引用保持有效，但是所有用于缓存磁盘数据的物理内存都将被取消提交.对 memmap 数组的任何视图也必须保持有效.

In other words, I'd like the reference to the memmap instance to remain valid, but all physical memory that's being used to cache the on-disk data to be uncommitted. Any views onto to the memmap array must also remain valid.

我希望将其用作诊断工具，以帮助将脚本的实际"内存需求与使用 memmap 引起的瞬态"需求分开.

I am hoping to use this as a diagnostic tool, to help separate "real" memory requirements of a script from "transient" requirements induced by the use of memmap.

我在RedHat上使用Python 2.7.

I'm using Python 2.7 on RedHat.

推荐答案

如果运行"pmap SCRIPT-PID"，则实际"内存显示为"[anon]"块，并且所有内存映射文件都显示为最后一列中的文件名.

If you run "pmap SCRIPT-PID", the "real" memory shows as "[ anon ]" blocks, and all memory-mapped files show up with the file name in the last column.

如果可以设法获得指向映射开始的指针并在其上调用madvise(ptr，length，MADV_DONTNEED)，则可以在C级别清除页面.

Purging the pages is possible at C level, if you manage to get ahold of the pointer to the beginning of the mapping and call madvise(ptr, length, MADV_DONTNEED) on it, but it's going to be cludgy.

这篇关于清除numpy.memmap的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！