在不增加 VmSize 的情况下增加虚拟内存 [英] Increase of virtual memory without increse of VmSize

问题描述

我在 Google 和这个网站上搜索了我的问题，但我仍然不明白解决方案.

I searched for my problem in Google and at this site but i still don't understand the solution.

我有一段 MPI 程序，其中 RECV 一些数据.程序在大数组上崩溃并出现虚拟内存不足的错误，所以我开始考虑 /proc/self/status 文件.

I have piece of MPI program which RECV some data. Program crashes on big arrays with error of insufficient virtual memory, and so i started to consider /proc/self/status file.

在 MPI_RECV 之前是:

Name:   model.exe                                                               
VmPeak:   841640 kB
VmSize:   841640 kB
VmHWM:     15100 kB
VmRSS:     15100 kB
VmData:   760692 kB

之后:

Name:   model.exe                                                            
VmPeak:   841640 kB
VmSize:   841640 kB
VmHWM:    719980 kB
VmRSS:    719980 kB
VmData:   760692 kB

我在 Ubuntu 上对其进行了测试，通过系统监视器我看到内存在增加.但是我很困惑 VmSize(和 VmPeak)参数没有变化.

I test it on Ubuntu and through System Monitor i saw this memory increasing. But i was confused that there are no changes in VmSize(and VmPeak) parameters.

问题是——实际内存使用的指标是什么?

And the question is - what is the indicator of real memory usage?

这是否意味着，真正的指标是VmRSS?(而 VmSize 只分配了内存，还没有使用)

Does it mean, that true indicator is VmRSS? (and VmSize is only allocated but still not used memory)

推荐答案

(你的问题可能的解决方案是最后一段)

(The possible solution to your problem is the last paragraph)

在大多数具有虚拟内存的现代操作系统上，内存分配是一个两阶段的过程.首先，进程的一部分虚拟地址空间被保留，进程的虚拟内存大小(VmSize)相应增加.这会在所谓的进程页表中创建条目.页面最初与物理内存帧无关，即实际上没有使用物理内存.每当实际读取或写入此分配部分的某些部分时，就会发生页面错误，并且操作系统会从物理内存中安装(映射)一个空闲页面.这会增加进程的驻留集大小 (VmRSS).当一些其他进程需要内存时，操作系统可能会将一些不经常使用的页面的内容(不经常使用的页面"的定义高度依赖于实现)存储到一些持久性存储(大多数情况下是硬盘驱动器，或者通常存储到交换设备)) 然后取消映射.此过程会减少 RSS，但保持 VmSize 不变.如果稍后访问该页面，则会再次发生页面错误并将其带回.仅当释放虚拟内存分配时，虚拟内存大小才会减小.请注意，VmSize 也计算内存映射文件(即可执行文件和它链接到的所有共享库或其他显式映射文件)和共享内存块.

Memory allocation on most modern operating systems with virtual memory is a two-phase process. First, a portion of the virtual address space of the process is reserved and the virtual memory size of the process (VmSize) increases accordingly. This creates entries in the so-called process page table. Pages are initially not associated with phyiscal memory frames, i.e. no physical memory is actually used. Whenever some part of this allocated portion is actually read from or written to, a page fault occurs and the operating system installs (maps) a free page from the physical memory. This increases the resident set size of the process (VmRSS). When some other process needs memory, the OS might store the content of some infrequently used page (the definition of "infrequently used page" is highly implementation-dependent) to some persistent storage (hard drive in most cases, or generally to the swap device) and then unmap up. This process decreases the RSS but leaves VmSize intact. If this page is later accessed, a page fault would again occur and it will be brought back. The virutal memory size only decreases when virtual memory allocations are freed. Note that VmSize also counts for memory mapped files (i.e. the executable file and all shared libraries it links to or other explicitly mapped files) and shared memory blocks.

进程中有两种通用类型的内存 - 静态分配的内存和堆内存.静态分配的内存保留所有常量和全局/静态变量.它是数据段的一部分，其大小由 VmData 指标显示.数据段还承载部分程序堆，其中分配了动态内存.数据段是连续的，即它从某个位置开始，向上向堆栈增长(从一个非常高的地址开始，然后向下增长).数据段中的堆的问题在于它由一个特殊的堆分配器管理，该分配器负责将连续的数据段细分为更小的内存块.另一方面，在Linux中动态内存也可以通过直接映射虚拟内存来分配.这通常只用于大分配以节省内存，因为它只允许分配页面大小的倍数(通常为 4 KiB)的内存.

There are two generic types of memory in a process - statically allocated memory and heap memory. The statically allocated memory keeps all constants and global/static variables. It is part of the data segment, whose size is shown by the VmData metric. The data segment also hosts part of the program heap, where dynamic memory is being allocated. The data segment is continuous, i.e. it starts at a certain location and grows upwards towards the stack (which starts at a very high address and then grows downwards). The problem with the heap in the data segment is that it is managed by a special heap allocator that takes care of subdividing the contiguous data segment into smaller memory chunks. On the other side, in Linux dynamic memory can also be allocated by directly mapping virtual memory. This is usually done only for large allocations in order to conserve memory, since it only allows memory in multiples of the page size (usually 4 KiB) to be allocated.

堆栈也是大量内存使用的重要来源，尤其是在自动(堆栈)存储中分配大数组时.堆栈从可用虚拟地址空间的最顶部附近开始并向下增长.在某些情况下，它可能会到达数据段的顶部，或者可能会到达其他一些虚拟分配的末尾.坏事就会发生.堆栈大小在 VmStack 指标和 VmSize 中计算.可以这样概括:

The stack is also an important source of heavy memory usage, especially if big arrays are allocated in the automatic (stack) storage. The stack starts near the very top of the usable virtual address space and grows downwards. In some cases it could reach the top of the data segment or it could reach the end of some other virtual allocation. Bad things happen then. The stack size is accounted in the VmStack metric and also in the VmSize. One can summarise it as so:

• VmSize 占所有虚拟内存分配(文件映射、共享内存、堆内存、任何内存)，并且几乎每次分配新内存时都会增长.几乎，因为如果新的堆内存分配是在数据段中释放的旧分配的位置进行的，则不会分配新的虚拟内存.每当释放虚拟分配时，它就会减少.VmPeak 跟踪 VmSize 的最大值 - 它只能随时间增加.
• VmRSS随着内存被访问而增长，随着内存被调出到交换设备而减少.
• VmData 随着堆的数据段部分被使用而增长.它几乎不会缩小，因为当前的堆分配器会保留已释放的内存以备将来分配需要时使用.

• VmSize accounts for all virtual memory allocations (file mappings, shared memory, heap memory, whatever memory) and grows almost every time new memory is being allocated. Almost, because if the new heap memory allocation is made in the place of a freed old allocation in the data segment, no new virtual memory would be allocated. It decreses whenever virtual allocations are being freed. VmPeak tracks the max value of VmSize - it could only increase in time.
• VmRSS grows as memory is being accessed and decreases as memory is paged out to the swap device.
• VmData grows as the data segment part of the heap is being utilised. It almost never shrinks as current heap allocators keep the freed memory in case future allocations need it.

如果您在使用 InfiniBand 或其他基于 RDMA 的结构的集群上运行，则另一种内存会发挥作用 - 锁定(注册)内存 (VmLck).这是不允许被调出的内存.它如何增长和缩小取决于 MPI 实现.有些人从不注销已经注册的块(有关原因的技术细节太复杂，无法在此处描述)，其他人这样做是为了更好地使用虚拟内存管理器.

If you are running on a cluster with InfiniBand or other RDMA-based fabrics, another kind of memory comes into play - the locked (registered) memory (VmLck). This is memory which is not allowed to be paged out. How it grows and shrinks depends on the MPI implementation. Some never unregister an already registered block (the technical details about why are too complex to be described here), others do so in order to play better with the virtual memory manager.

在您的情况下，您说您遇到了虚拟内存大小限制.这可能意味着此限制设置得太低，或者您遇到了操作系统强加的限制.首先，Linux(和大多数 Unix)有办法通过 ulimit 机制施加人为的限制.在 shell 中运行 ulimit -v 会告诉您虚拟内存大小的限制(以 KiB 为单位).您可以使用 ulimit -v 设置限制.这仅适用于由当前 shell 生成的进程及其子、孙等.如果要在远程节点上启动，您需要指示 mpiexec(或 mpirun)将此值传播到所有其他进程.如果您在 LSF、Sun/Oracle Grid Engine、Torque/PBS 等工作负载管理器的控制下运行程序，则有一些作业参数可以控制虚拟内存大小限制.最后但同样重要的是，32 位进程通常限制为 2 GiB 的可用虚拟内存.

In your case you say that you are running into a virtual memory size limit. This could mean that this limit is set too low or that you are running into an OS-imposed limits. First, Linux (and most Unixes) have means to impose artificial restrictions through the ulimit mechanism. Running ulimit -v in the shell would tell you what the limit on the virtual memory size is in KiB. You can set the limit using ulimit -v <value in KiB>. This only applies to processes spawned by the current shell and to their children, grandchilren and so on. You need to instruct mpiexec (or mpirun) to propagate this value to all other processes, if they are to be launched on remote nodes. if you are running your program under the control of some workload manager like LSF, Sun/Oracle Grid Engine, Torque/PBS, etc., there are job parameters which control the virtual memory size limit. And last but not least, 32-bit processes are usually restricted to 2 GiB of usable virtual memory.

这篇关于在不增加 VmSize 的情况下增加虚拟内存的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！