NUMA:如何检查C ++数组分配在RAM的哪一部分? [英] NUMA: How to check in which part of RAM a C++ array is allocated?

问题描述

我有一台服务器，带有2个CPU和64GB的RAM，每个CPU 32GB.

I have a server with 2 CPU's and 64GB of RAM, 32GB per CPU.

我知道每个CPU都有它自己的RAM，请称它们为RAM1和RAM2.我想让我的程序知道它在哪个RAM(RAM1或RAM2)上分配数据.

I know that each CPU has it's own part of RAM, lets call them RAM1 and RAM2. I would like to make my program know on which RAM (RAM1 or RAM2) it allocates it's data.

我试图检查指针值:

  // put the thread at i-th CPU, using pthread_setaffinity_np
TData *a = new TData[N];
...
cout << "CPU = " << i << " adress = " << a << endl; 

，但输出看起来是随机的.我想那是因为地址是虚拟的.虚拟内存地址和部分RAM之间有对应关系吗?

but the output looks random. I suppose that is because addresses are virtual. Is there any correspondence between virtual memory addresses and part of the RAM?

如何检查我的阵列"a"分配在哪个RAM中?

推荐答案

在此处回答了您的问题.我只想添加一些评论.

Your question is answered here. I would only like to add some comments.

请注意，调用new []实际上并不分配物理内存.在现代操作系统上，这只会导致开始进行匿名内存映射.匿名映射不对应于文件系统中的文件，而是由交换(如果有)支持.最初，整个区域指向内核中包含所有零的只读页面.仅当您实际写入新分配的内存时，才会安装新的内存页面，该页面将零页面替换为访问地址所在的页面范围.这就是为什么我们说零页被写时复制(或CoW)映射到进程的虚拟地址空间.默认策略是尝试在访问内存区域的线程所在的同一NUMA节点上分配新页面.这被称为第一接触" NUMA策略.如果该NUMA节点上没有足够的内存，则会在其他节点上为该页面分配足够的可用内存.小分配也有可能最终由C库内存分配器malloc()管理(C ++运算符new []调用malloc()以便进行实际的内存分配)，从而在更大的区域(称为竞技场)内结束.在这种情况下，即使在您写入新分配的内存之前，页面也可能已经存在于物理内存中.

Note that calling new [] does not actually allocate physical memory. On modern operating systems this only results in an anonymous memory mapping begin made. Anonymous mappings do not correspond to files in the file system but rather are backed by the swap (if any). Initially the whole area points to a read-only page in the kernel that contains all zeros. Only when you actually write to the newly allocated memory, a new memory page is installed, which replaces the zero page for the page range where the accessed address falls. This is why we say that the zero page is copy-on-write (or CoW) mapped to the virtual address space of the process. The default policy is to try to allocate the new page on the same NUMA node, where the thread that accessed the memory area runs. This is called a "first-touch" NUMA policy. If there is not enough memory on that NUMA node, the page is allocated on some other node with enough free memory. It is also possible for small allocations to end up inside a bigger area (called arena), managed by the C library memory allocator malloc() (the C++ operator new [] calls malloc() in order to do the actual memory allocation). In this case the pages might be already present in the physical memory even before you write to the newly allocated memory.

Linux有一个讨厌的习惯，即在交换时不保留内存区域的NUMA关联.也就是说，如果在NUMA节点0上分配了一个页面，然后换出然后又换回了该页面，则不能保证该页面不会放在NUMA节点1上.这将产生以下问题:我的内存分配在哪里"有点棘手，因为连续换出后再换入很容易使您从move_pages()中获得的结果无效，而仅需不到一秒钟的时间.因此，该问题仅在以下两种特殊情况下才有意义:

Linux has the nasty habit of not preserving the NUMA association of memory areas when it swaps. That is, if a page was allocated on NUMA node 0, then swapped out and then swapped back in, there is no guarantee that the page won't be placed on NUMA node 1. This makes the question "where is my memory allocated" a bit tricky, since a consecutive swap-out followed by a swap-in could easily invalidate the result that you get from move_pages() just a fraction of a second ago. Therefore the question only makes sense in the following two special cases:

• 您显式锁定了内存区域:可以使用mlock(2)系统调用来告诉OS不要从进程虚拟地址空间交换特定范围；
• 您的系统没有活动的交换区域::这可以防止操作系统完全将页面从主内存移出或移回主内存.

• you explicitly lock the memory region: one could use the mlock(2) system call in order to tell the OS not to swap a particular range from the process virtual address space;
• your system has no active swap areas: this prevents the OS altogether from moving the pages out from and back into the main memory.

这篇关于NUMA:如何检查C ++数组分配在RAM的哪一部分?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！