在大型阵列上使用VirtualAlloc保留与提交+保留内存的优势 [英] Benefits of reserving vs. committing+reserving memory using VirtualAlloc on large arrays

问题描述

我正在编写一个C ++程序，该程序实际上可用于非常大的数组.在Windows上，我使用VirtualAlloc将内存分配给我的阵列.现在，我完全了解了使用VirutalAlloc保留和提交内存之间的区别.但是，我想知道将内存逐页提交到保留区域是否有任何好处.特别是MSDN( http://msdn.microsoft.com/zh-cn/library/windows/desktop/aa366887(v=vs.85).aspx )包含MEM_COMMIT选项的以下说明:

I am writing a C++ program that essentially works with very large arrays. On Windows, I am using VirtualAlloc to allocate memory to my arrays. Now I fully understand the difference between reserving and committing memory using VirutalAlloc; however, I am wondering whether there is any benefit in committing memory page-by-page to a reserved region. In particular, MSDN (http://msdn.microsoft.com/en-us/library/windows/desktop/aa366887(v=vs.85).aspx) contains the following explanation for the MEM_COMMIT option:

除非/直到实际访问虚拟地址，否则不会分配实际的物理页面.

Actual physical pages are not allocated unless/until the virtual addresses are actually accessed.

我的实验证实了这一点:我可以保留并提交数GB的内存，而不会增加进程的内存使用率(如任务管理器中所示)；仅当我实际访问内存时，才会分配实际内存.

My experiments confirm this: I can reserve and commit several GB of memory wihtout increasing memory usage of my process (as shown in Task Manager); actual memory gets allocated only when I actually access memory.

现在，我看到了很多示例，它们争辩说应该保留很大一部分地址空间，然后逐页提交内存(或在更大的块中，具体取决于应用程序的逻辑).但是，如上文所述，似乎在访问内存之前并没有提交内存.因此，我想知道逐页提交内存是否有任何真正的好处.实际上，由于要实际提交内存的许多系统调用，一页一页地提交内存实际上可能会使我的程序变慢.如果我一次提交整个区域，那么我只为一个系统调用付费，但是内核似乎足够聪明，可以只分配我实际使用的内存.

Now I saw quite a few examples arguing that one should reserve a large portion of the address space and then commit memory page-by-page (or in some larger blocks, depending on the app's logic). As explained above, however, memory does not seem to be committed before one accesses it; thus, I'm wondering whether there is any real benefit in committing memory page-by-page. In fact, committing memory page-by-page might actually slow my program down due to many system calls for actually comitting memory. If I commit the entire region at once, I pay for just one system call, but the kernel seems to be smart enough to actually allocate only memory that I actually use.

如果有人可以向我解释哪种策略更好，我将不胜感激.

I would appreciate it if someone could explain to me which strategy is better.

推荐答案

区别在于，针对页面文件提交返回"内存.举个例子:

The difference is that commit "backs" the memory against the page file. To give an example:

1. 给出2GB的物理内存和2GB的交换空间(为此目的，假设使用固定大小的交换空间).
2. 保留6GB-确定.
3. 先提交2GB-好.
4. 提交剩余的4GB-失败.
5. 将交换文件扩展到8GB
6. 提交剩余的4GB-成功.

使用MEM_COMMIT的原因主要是为了抑制运行时错误(应用程序稳定性).如果您有一个按需提交页面的过程，那么如果超过可用的内存和交换数量，则始终有可能导致提交失败.当页面文件支持内存时，您有一个 strong 保证从现在开始直到释放该内存为止，都可以使用该内存.

The reason for using MEM_COMMIT would primarily be for runtime error suppression (app stability). If you have a process that commits pages on-demand then there's always a chance that a commit along-the-way could fail if it exceeds amount of memory+swap available. When memory has been backed by the page file then you have a strong guarantee that the memory is available for use from now until the point that you release it.

有很多理由可以选择一种方法，而我认为没有哪种完美的科学方法可以决定哪种方法.仅MEM_RESERVE仅需要非常大型的稀疏数组方案，例如:千兆字节数组，利用率最高为25-33％(一种用于加速哈希表的流行技术，等等).

There's a number of reasons to go one way or the other, and I don't think there's any perfect science to deciding which. MEM_RESERVE alone is only needed for very large sparse array scenarios, ex: multi-gigabyte array which has at most 25-33% utilization (a popular technique for accelerating hash tables, etc).

几乎所有其他区域都是灰色区域，您可能会选择其中任何一种方式-MEM_COMMIT预先使您自己的应用程序更加稳定，并且实际上将其优先于物理内存，而不是可能按需分配的竞争应用程序. (如果您先抓住内存，那么当物理内存耗尽时，您的应用程序将是最后一站).同时，如果您实际上并没有使用所有内存，那么最终可能会限制您的多任务处理能力客户的计算机或通过不断增长的页面文件造成不必要的磁盘空间浪费.

Almost everything else is gray area where you could probably go either way -- MEM_COMMIT up-front would make your own app a little more stable and essentially give it priority to physical ram over competing apps that might allocate on-demand. (if you grab the ram first then your app will be the last left standing when physical memory is exhausted) At the same time, if you're not actually using all that ram then you may end up limiting the multi-tasking potential of your client's machine or causing unnecessary wasted disk space via a growing page file.

这篇关于在大型阵列上使用VirtualAlloc保留与提交+保留内存的优势的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！