如何在支持用户模式NUMA的内存分配器中实现交错的页面分配? [英] How to implement interleaved page allocation in a user-mode NUMA-aware memory allocator?

问题描述

我正在为Linux构建一个用户模式的NUMA感知内存分配器.分配器在初始化期间会占用大量内存，每个NUMA节点一个内存.此后，通过从大块池中提供尽可能多的内存页面来满足用户请求的内存页面.

I am building a user-mode NUMA-aware memory allocator for linux. The allocator during its initialization grabs a large chunk of memory, one chunk per NUMA node. After this, memory pages requested by the user are met by giving as many memory pages from the large chunk pool.

如果用户要求n页，则可以很容易地从特定块中分配n页.但是现在我想实现一种交错分配策略，即用户从每个块中获得一页，循环最多n页.这带来了这些页面的虚拟地址不再连续的问题.

If the user asks for n pages, it is easy to give n pages from a particular chunk. But now I want to implement an interleaved allocation policy, where the user gets one page from each chunk, round-robin up to n pages. This brings up the problem of the virtual addresses of these pages no longer being contiguous.

Q1:有没有办法返回虚拟可寻址的连续内存?我能想到的唯一解决方案是使用智能"指针，该指针知道如何从一页跳到另一页.

Q1: Is there a way to return virtually addressable contiguous memory? The only solution I can think of is using a "smart" pointer who knows how to jump from one page to another.

我走这条路的原因之一是，我对Linux的MPOL_INTERLEAVE内存分配策略不满意，该策略的循环策略不严格(确定性).

One of the reasons I am walking this path is that I am not happy with the MPOL_INTERLEAVE memory allocation policy of linux whose round-robin policy is not strict (deterministic).

Q2:是否存在一种廉价的方法来知道给定虚拟地址范围映射到的页面和NUMA节点?更准确地说，我不了解如何通过阅读/proc/< proc_id>/numa_maps.

Q2: Is there an inexpensive way of knowing which page and NUMA node a given virtual address range is mapped to? More precisely I do not how to get fine-grained page-level information from reading /proc/< proc_id >/numa_maps .

谢谢您的回答.

推荐答案

A1. 虚拟连续内存并不意味着物理内存是连续的.在linux中，物理页面不是在malloc期间绑定到虚拟页面的，而是在第一页错误期间绑定的.

A1. Virtually contiguous memory does not imply that the physical memory is contiguous. In linux, physical pages are not bound to virtual pages during a malloc, but rather during the first page fault.

如果确实需要，您应该能够对页面进行预故障处理，以将它们绑定到特定的numa节点，以便使用默认分配策略创建严格的交错.

If you really wanted to, you should be able to pre-fault the pages to bind them to a particular numa node in order to create the strict interleaving using the default allocation policy.

例如

N - # numa nodes
PAGES - # pages in allocation

for(i=0; i < N; i++):
   pin current thread to node i
   for(p=i; p < PAGES; p += N)
      touch page p;

完成设置后，您可以展示预先交错的连续页面.

After you have that set up, you can dish out pre-interleaved contiguous pages.

Q2. 您可以使用<numaif.h>中的move_pages来确定虚拟地址的numa节点 并传递NULL作为目标节点.当前节点位置将在状态返回值中.

Q2. You can determine the numa node of a virtual address by using move_pages from <numaif.h> and passing NULL as the target node. The current node location will be in the status return value.

例如

int status[1];
move_pages(0, 1, &ptr_to_check,  NULL, status, 0);

这篇关于如何在支持用户模式NUMA的内存分配器中实现交错的页面分配?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！