Windows 32位虚拟内存页面映射问题 [英] Windows 32-bit virtual memory page mapping issue

问题描述

我从这里开始学习Windows 32位虚拟内存页面映射，

(我的目标是现代Windows版本，例如Vista，Win 7，Server 2003/2008 32位版本.)

http://blogs.msdn.com/ericlippert/archive/2009/06/08/out-of-memory-does-not-refer-to-physical-memory.aspx

两个困惑，

1. 用户空间虚拟内存空间通常限制为2G，但是物理磁盘页面存储可能比2G大得多.由于磁盘页面多于虚拟内存页面，因此理论上可以将一个以上的磁盘页面映射到一个虚拟地址页面.如果用户请求访问某个虚拟地址，如果将多个磁盘页面映射到一个虚拟地址页面，内存管理器如何知道应该访问哪个磁盘页面?

2. 我不知道为什么有限制，例如byte []数组必须使用连续的虚拟内存空间.我认为从理论上讲，即使我们仅分配500M虚拟空间地址，我们也可以重新使用该虚拟空间地址来继续映射/取消映射磁盘页面文件以消耗我们想要的最大数量，甚至大于2G.

有什么想法吗?

解决方案

字节(或任何其他)数组必须使用连续的地址空间，在这种情况下，连续的 virtual 地址空间. 这是一个可能使碎片化成为问题的领域，并且实际上由于内存虚拟化而加剧了这一问题.由于此类系统系统固有的各种重定向"以及有效的性能考虑会导致实际的物理地址空间分配成块(通常是页面)映射到虚拟地址空间.

因此，请求10个字节的虚拟地址空间块实际上可能会导致整个4K页面被保留和映射.由于页面内的物理内存必须是连续的，因此可能导致"4k"虚拟地址空间整个被阻塞".
可以在一个页面中放置多个小分配(好的内存管理器将尝试实现此目的)，但是实际上，除了严格要求的地址空间之外，实际上还保留了较少的地址空间.考虑在页面的开始处分配单个字节，然后分配4K-2个字节，然后分配另一个单个字节.这会(有效地)占据整个页面.
考虑是否不再需要中间分配，并因此释放了中间分配.在释放"top"和"tail"值或将其移动到其他位置之前，已在虚拟地址空间中创建了一个间隙，该间隙只能由<大小为4K-1字节. 如果发生了足够多的这类事情，则虚拟地址空间中的连续区域收缩的速度将比实际总真正使用的存储器的收缩速度快得多.

您是对的，用户没有什么可以阻止您的，将您的地址空间(限制在32位内)映射到CPU/OS支持的更大的磁盘或内存空间.某些芯片通过诸如 PAE 之类的机制，利用超过4GB的物理地址空间来实现这一目标. >

Windows本身提供了一个API，用于处理更改地址空间的映射以使另一个"窗口"进入更大的池的大多数方面(通过PAE，运行WoW64，磁盘或混合方式).这称为 AWE .但是像这样的机制已经存在了多年(例如那些使用常规记忆来记住EMS的人或分段内存的日子.

即使没有CPU和OS支持，您仍然可以通过多种技术手动完成此操作(见下文).

许多有趣的Raymond Chen处理了Windows中许多更复杂的方面.

• 相关文章的有用列表
• 他跟进了一些细节，并链接到您实际上如何操作的方式您不是唯一占用进程中地址空间的人

I am learning from here about Windows 32-bit virtual memory page mapping,

(I am targeting modern Windows versions, like Vista, Win 7, Server 2003/2008 32-bit versions.)

http://blogs.msdn.com/ericlippert/archive/2009/06/08/out-of-memory-does-not-refer-to-physical-memory.aspx

Two confusions,

1. user space virtual memory space is limited to 2G normally, but physical disk page storage could be much larger than 2G. Since there are more disk pages than virtual memory pages, so in theory more than one disk pages could be mapped to one virtual address page. If user request to access some virtual address, how did memory manager know which disk page should be accessed if more than one disk pages are mapped to one virtual address page?

2. I do not know why there is restrictions like byte[] array must use continuous virtual memory space. I think in theory even if we only allocate 500M virtual space address, we can re-use such virtual space address to continue to map/unmap disk page file to consume as much as we want, even larger than 2G.

Any ideas?

解决方案

byte (or any other) arrays must use contiguous address space, in this case contiguous virtual address space. This is an area that can make fragmentation becomes an issue, and is actually exacerbated by virtualization of memory. Since the various 'redirections' inherent in a such a system system and the performance considerations in making it efficient lead to actual allocation of physical address space mapped to virtual address space in chunks (commonly pages).

So requesting a 10 byte chunk of virtual address space might actually lead to a whole 4K page being reserved and mapped. Since the physical memory within the page must be contiguous this can lead to a whole 4K of virtual address space being 'blocked off'.
Multiple small allocations can be placed within one page (and good memory managers will try to achieve this) but none the less address space has in effect been reserved over and above what was strictly required. Consider the allocation of a single byte at the start f a page, then 4K - 2 bytes followed by another single byte. This occupies (efficiently) the whole page.
Consider if the middle allocation is no longer necessary, and is thus freed. Until the 'top' and 'tail' values are either freed or moved elsewhere a gap has been created in the virtual address space which can only be filled by something < 4K-1 bytes in size. If sufficient of these sorts of things occur the contiguous regions in virtual address space shrink much faster than the actual total really used memory does.

You are correct in that there is nothing stopping you, the user, mapping your (limited in 32bit land) address space to the much larger disk or memory space that the CPU/OS supports . Some chips make this possible with more than 4GB of physical address space via mechanisms like PAE.

Windows itself provides an API to deal with most aspects of 'changing the mapping of your address space to get a different 'window' onto a wider pool (be it via things like PAE, running WoW64, disk or a mixture). This is called AWE. But mechanisms like this have existed for years (as those who remember the days of EMS with conventional memory or indeed the days of segmented memory.

Even without CPU and OS support you can still do it yourself by hand by a variety of techniques (see below).

A great deal of the more complex aspects of this in windows were dealt with by ever interesting Raymond Chen.

• A useful list of the relevant articles
• He follows it up with some specifics and links to how you can actually allocate more than 2GB of memory
• Also worth remembering is that you aren't the only person consuming address space in your process

这篇关于Windows 32位虚拟内存页面映射问题的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！