增加虚拟内存而不增加VmSize [英] Increase of virtual memory without increse of VmSize

问题描述

我有一块 MPI 程式 RECV 一些资料。程序在虚拟内存不足的情况下在大阵列上崩溃，所以我开始考虑 / proc / self / status 文件。

在 MPI_RECV 之前它是：

 名称：model。 
 VmPeak：841640 kB 
 VmSize：841640 kB 
 VmHWM：15100 kB 
 VmRSS：15100 kB 
 VmData：760692 kB 
  
 
 
名称：model .exe 
 VmPeak：841640 kB 
 VmSize：841640 kB 
 VmHWM：719980 kB 
 VmRSS：719980 kB 
 VmData：760692 kB 
  
我在Ubuntu上测试它，并通过系统监视器，我看到这个内存在增加。但是我很困惑， VmSize （和 VmPeak ）参数没有变化。
 
 
 
问题是 - 真实内存使用情况的指标是什么？  
 
 
这是否意味着真正的指标是 VmRSS ？ （并且 VmSize 仅被分配，但仍未使用内存）       （可能的解决方案是最后一段） 
 
 
大多数具有虚拟内存的现代操作系统的内存分配是一个两阶段过程。首先，进程的虚拟地址空间的一部分被保留，进程的虚拟内存大小（ VmSize ）相应地增加。这会在所谓的进程页表中创建条目。页面最初不与物理内存帧关联，即没有实际使用物理内存。只要实际读取或写入此分配部分的某个部分，就会发生页面错误，操作系统会从物理内存中安装（映射）空闲页面。这会增加进程的驻留集大小（ VmRSS ）。当某些其他进程需要内存时，操作系统可能会将某些不常使用的页面的内容（不常使用的页面的定义依赖于高度实现）存储到某些持久性存储器（大多数情况下为硬盘驱动器，或者通常交换设备），然后取消映射。这个过程减少了RSS，但是保持 VmSize 不变。如果稍后访问此页面，则会再次发生页面错误并将其恢复。只有释放虚拟内存分配时，虚拟内存大小才会减小。请注意， VmSize 对内存映射文件（即可执行文件及其链接的所有共享库或其他明确映射的文件）和共享内存块也计数。 b 
 $ b 
在进程中有两种通用类型的内存 - 静态分配的内存和堆内存。静态分配的内存保留所有常量和全局/静态变量。它是数据段的一部分，其大小由 VmData 度量标准显示。数据段还承载程序堆的一部分，在这里分配动态内存。数据段是连续的，即它从一个特定的位置开始向上朝堆栈方向增长（从一个非常高的地址开始，然后向下增长）。数据段中堆的问题在于它由一个特殊的堆分配器管理，该堆分配器负责将连续数据段细分为更小的内存块。另一方面，在Linux中，动态内存也可以通过直接映射虚拟内存来分配。这通常是为了节约内存而进行的大分配，因为它只允许分配多倍页面大小的内存（通常为4 KiB）。
 
 
 
堆栈也是大量内存使用的重要来源，特别是在自动（堆栈）存储中分配大数组时。堆栈开始于可用虚拟地址空间的最顶端并向下增长。在某些情况下，它可能会达到数据段的顶部，或者可能会达到某些其他虚拟分配的末尾。那时坏事发生了。堆栈大小在 VmStack 度量标准中以及 VmSize 中。 
可以总结为： 
 
 
 
 
  VmSize 所有帐户虚拟内存分配（文件映射，共享内存，堆内存，任何内存），几乎每次新内存分配时都会增长。几乎是因为如果新的堆内存分配是在数据段中释放旧分配的地方进行的，则不会分配新的虚拟内存。只要虚拟分配被释放，它就会减少。  VmPeak 跟踪 VmSize 的最大值 - 它只能在时间上增加。
 
 
  VmRSS 随着内存被访问而增长，并随着内存被分页到交换设备而减少。
 
 
  VmData 随着堆的数据段部分被使用而增长。它几乎不会缩小，因为当前的堆分配器保留释放的内存，以防将来的分配需要它。
 
 
 
 
 
如果您正在群集上运行InfiniBand或其他基于RDMA的光纤网络，另一种内存进入游戏 - 锁定（注册）的内存（ VmLck ）。这是不允许被分页的内存。它如何增长和缩小取决于MPI的实施。有些人从未取消注册一个已经注册的模块（关于为什么这些模块太复杂的技术细节不能在这里描述），有些模块是为了更好地利用虚拟内存管理器而做的。
 
 
 
在你的情况下，你说你正在运行一个虚拟内存大小限制。这可能意味着此限制设置得太低，或者您正在运行OS限制。首先，Linux（以及大多数Unix）有通过 ulimit 机制施加人为限制的手段。在shell中运行 ulimit -v 会告诉你在KiB中虚拟内存大小的限制。您可以使用 ulimit -v 来设置限制。这仅适用于当前shell以及子孙，孙辈等产生的进程。您需要指示 mpiexec （或 mpirun ）将此值传播到所有其他进程，如果它们要启动在远程节点上。如果您在像LSF，Sun / Oracle Grid Engine，Torque / PBS等工作负载管理器的控制下运行程序，则存在控制虚拟内存大小限制的作业参数。最后但并非最不重要的是，32位进程通常限制在2 GiB的可用虚拟内存。
 
I searched for my problem in Google and at this site but i still don't understand the solution.

I have piece of MPI program which RECV some data. Program crashes on big arrays with error of insufficient virtual memory, and so i started to consider /proc/self/status file.

Before MPI_RECV it was:
Name:   model.exe                                                               
VmPeak:   841640 kB
VmSize:   841640 kB
VmHWM:     15100 kB
VmRSS:     15100 kB
VmData:   760692 kB
And after:
Name:   model.exe                                                            
VmPeak:   841640 kB
VmSize:   841640 kB
VmHWM:    719980 kB
VmRSS:    719980 kB
VmData:   760692 kB
I test it on Ubuntu and through System Monitor i saw this memory increasing. But i was confused that there are no changes in VmSize(and VmPeak) parameters.

And the question is - what is the indicator of real memory usage? 

Does it mean, that true indicator is VmRSS? (and VmSize is only allocated but still not used memory)
解决方案
(The possible solution to your problem is the last paragraph)

Memory allocation on most modern operating systems with virtual memory is a two-phase process. First, a portion of the virtual address space of the process is reserved and the virtual memory size of the process (VmSize) increases accordingly. This creates entries in the so-called process page table. Pages are initially not associated with phyiscal memory frames, i.e. no physical memory is actually used. Whenever some part of this allocated portion is actually read from or written to, a page fault occurs and the operating system installs (maps) a free page from the physical memory. This increases the resident set size of the process (VmRSS). When some other process needs memory, the OS might store the content of some infrequently used page (the definition of "infrequently used page" is highly implementation-dependent) to some persistent storage (hard drive in most cases, or generally to the swap device) and then unmap up. This process decreases the RSS but leaves VmSize intact. If this page is later accessed, a page fault would again occur and it will be brought back. The virutal memory size only decreases when virtual memory allocations are freed. Note that VmSize also counts for memory mapped files (i.e. the executable file and all shared libraries it links to or other explicitly mapped files) and shared memory blocks.

There are two generic types of memory in a process - statically allocated memory and heap memory. The statically allocated memory keeps all constants and global/static variables. It is part of the data segment, whose size is shown by the VmData metric. The data segment also hosts part of the program heap, where dynamic memory is being allocated. The data segment is continuous, i.e. it starts at a certain location and grows upwards towards the stack (which starts at a very high address and then grows downwards). The problem with the heap in the data segment is that it is managed by a special heap allocator that takes care of subdividing the contiguous data segment into smaller memory chunks. On the other side, in Linux dynamic memory can also be allocated by directly mapping virtual memory. This is usually done only for large allocations in order to conserve memory, since it only allows memory in multiples of the page size (usually 4 KiB) to be allocated.

The stack is also an important source of heavy memory usage, especially if big arrays are allocated in the automatic (stack) storage. The stack starts near the very top of the usable virtual address space and grows downwards. In some cases it could reach the top of the data segment or it could reach the end of some other virtual allocation. Bad things happen then. The stack size is accounted in the VmStack metric and also in the VmSize.
One can summarise it as so:


VmSize accounts for all virtual memory allocations (file mappings, shared memory, heap memory, whatever memory) and grows almost every time new memory is being allocated. Almost, because if the new heap memory allocation is made in the place of a freed old allocation in the data segment, no new virtual memory would be allocated. It decreses whenever virtual allocations are being freed. VmPeak tracks the max value of VmSize - it could only increase in time.
VmRSS grows as memory is being accessed and decreases as memory is paged out to the swap device.
VmData grows as the data segment part of the heap is being utilised. It almost never shrinks as current heap allocators keep the freed memory in case future allocations need it.


If you are running on a cluster with InfiniBand or other RDMA-based fabrics, another kind of memory comes into play - the locked (registered) memory (VmLck). This is memory which is not allowed to be paged out. How it grows and shrinks depends on the MPI implementation. Some never unregister an already registered block (the technical details about why are too complex to be described here), others do so in order to play better with the virtual memory manager.

In your case you say that you are running into a virtual memory size limit. This could mean that this limit is set too low or that you are running into an OS-imposed limits. First, Linux (and most Unixes) have means to impose artificial restrictions through the ulimit mechanism. Running ulimit -v in the shell would tell you what the limit on the virtual memory size is in KiB. You can set the limit using ulimit -v <value in KiB>. This only applies to processes spawned by the current shell and to their children, grandchilren and so on. You need to instruct mpiexec (or mpirun) to propagate this value to all other processes, if they are to be launched on remote nodes. if you are running your program under the control of some workload manager like LSF, Sun/Oracle Grid Engine, Torque/PBS, etc., there are job parameters which control the virtual memory size limit. And last but not least, 32-bit processes are usually restricted to 2 GiB of usable virtual memory.

                        
这篇关于增加虚拟内存而不增加VmSize的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！