使用 pmap 分析进程的内存映射.[堆] [英] Analyzing memory mapping of a process with pmap. [stack]

问题描述

我试图了解堆栈在 Linux 中的工作原理.我读了 AMD64 ABI关于堆栈和进程初始化的部分，并且不清楚堆栈应该如何映射.这是相关的引用(3.4.1):

I'm trying to understand how stack works in Linux. I read AMD64 ABI sections about stack and process initialization and it is not clear how the stack should be mapped. Here is the relevant quote (3.4.1):

堆栈状态

本节描述了 exec (BA_OS) 为其创建的机器状态新流程.

This section describes the machine state that exec (BA_OS) creates for new processes.

和

未指定数据和堆栈段是否初始是否映射了执行权限.需要的应用在堆栈或数据段上执行代码应该正确预防措施，例如，通过调用 mprotect().

It is unspecified whether the data and stack segments are initially mapped with execute permissions or not. Applications which need to execute code on the stack or data segments should take proper precautions, e.g., by calling mprotect().

所以我可以从上面的引号中推断出堆栈是映射的(未指定是否使用 PROT_EXEC 来创建映射).映射也是由 exec 创建的.

So I can deduce from the quotes above that the stack is mapped (it is unspecified if PROT_EXEC is used to create the mapping). Also the mapping is created by exec.

问题是主线程"的堆栈是否使用MAP_GROWSDOWN |MAP_STACK 映射或什至通过 sbrk?

The question is whether the "main thread"'s stack uses MAP_GROWSDOWN | MAP_STACK mapping or maybe even via sbrk?

查看pmap -x 栈被标记为[stack] as

Looking at pmap -x <pid> the stack is marked with [stack] as

00007ffc04c78000     132      12      12 rw---   [ stack ]

创建映射为

mmap(NULL, 4096,
     PROT_READ | PROT_WRITE,
     MAP_ANONYMOUS | MAP_PRIVATE | MAP_STACK,
     -1, 0);

简单地创建匿名映射，如 pmap -x <pid> as

simply creates anonymous mapping as that is shown in pmap -x <pid> as

00007fb6e42fa000       4       0       0 rw---   [ anon ]

推荐答案

我可以从上面的引号中推断出栈是映射的

I can deduce from the quotes above that the stack is mapped

字面意思就是分配了内存.即存在从这些虚拟地址到物理页面的逻辑映射.我们知道这一点是因为您可以在 _start 中使用 push 或 call 指令，而无需从用户空间进行系统调用来分配堆栈.

That literally just means that memory is allocated. i.e. that there is a logical mapping from those virtual addresses to physical pages. We know this because you can use a push or call instruction in _start without making a system call from user-space to allocate a stack.

事实上，x86-64 System V ABI 指定 argc、argv 和 envp 在进程启动时位于堆栈上.

In fact the x86-64 System V ABI specifies that argc, argv, and envp are on the stack at process startup.

问题是主线程"的堆栈是否使用MAP_GROWSDOWN |MAP_STACK 映射或什至通过 sbrk?

The question is whether the "main thread"'s stack uses MAP_GROWSDOWN | MAP_STACK mapping or maybe even via sbrk?

ELF 二进制加载器为主线程的堆栈设置 _GROWSDOWN 标志，但不设置 MAP_STACK 标志.这是内核内部的代码，它不通过常规的mmap系统调用接口.

The ELF binary loader sets the _GROWSDOWN flag for the main thread's stack, but not the MAP_STACK flag. This is code inside the kernel, and it does not go through the regular mmap system call interface.

(用户空间中没有任何东西使用 mmap(MAP_GROWSDOWN) 所以通常主线程堆栈是唯一具有 VM_GROWSDOWN 标志的映射内核内部.)

(Nothing in user-space uses mmap(MAP_GROWSDOWN) so normally the main thread stack is the only mapping that have the VM_GROWSDOWN flag inside the kernel.)

用于堆栈的虚拟内存区域 (VMA) 的标志的内部名称称为 VM_GROWSDOWN.如果您有兴趣，以下是用于主线程堆栈的所有标志:VM_GROWSDOWN、VM_READ、VM_WRITE、VM_MAYREAD、VM_MAYWRITE 和 VM_MAYEXEC.此外，如果指定 ELF 二进制文件具有可执行堆栈(例如，通过使用 gcc -z execstack 编译)，则还使用 VM_EXEC 标志.请注意，在支持向上增长的堆栈的体系结构上，如果内核是在定义了 CONFIG_STACK_GROWSUP 的情况下编译的，则使用 VM_GROWSUP 而不是 VM_GROWSDOWN.在 Linux 内核中指定这些标志的代码行可以找到 此处.

The internal name of the flag that is used for the virtual memory aree (VMA) of the stack is called VM_GROWSDOWN. In case you're interested, here are all the flags that are used for the main thread's stack: VM_GROWSDOWN, VM_READ, VM_WRITE, VM_MAYREAD, VM_MAYWRITE, and VM_MAYEXEC. In addition, if the ELF binary is specified to have an executable stack (e.g., by compiling with gcc -z execstack), the VM_EXEC flag is also used. Note that on architectures that support stacks that grow upwards, VM_GROWSUP is used instead of VM_GROWSDOWN if the kernel was compiled with CONFIG_STACK_GROWSUP defined. The line of code where these flags are specified in the Linux kernel can be found here.

/proc/.../maps 和 pmap 不使用 VM_GROWSDOWN - 它们依赖于地址比较.因此，他们可能无法准确确定主线程堆栈占用的虚拟地址空间的确切范围(请参阅 示例).另一方面，/proc/.../smaps 查找 VM_GROWSDOWN 标志并将具有此标志的每个内存区域标记为 gd.(虽然它似乎忽略了 VM_GROWSUP.)

/proc/.../maps and pmap don't use the VM_GROWSDOWN - they rely on address comparison instead. Therefore they may not be able to determine exactly the exact range of the virtual address space that the main thread's stack occupies (see an example). On the other hand, /proc/.../smaps looks for the VM_GROWSDOWN flag and marks each memory region that has this flag as gd. (Although it seems to ignore VM_GROWSUP.)

所有这些工具/文件都忽略了 MAP_STACK 标志.事实上，整个 Linux 内核都忽略了这个标志(这可能是程序加载器没有设置它的原因.)用户空间只会在内核确实想要启动的情况下传递它以备将来证明专门处理线程堆栈分配.

All of these tools/files ignore the MAP_STACK flag. In fact, the whole Linux kernel ignores this flag (which is probably why the program loader doesn't set it.) User-space only passes it for future-proofing in case the kernel does want to start treating thread-stack allocations specially.

sbrk 在这里没有意义；堆栈与中断"不连续，并且 brk 堆无论如何向上向 增长.Linux 将堆栈放在非常靠近虚拟地址空间顶部的位置.所以当然主堆栈不能用(内核中的等价物)sbrk 分配.

sbrk makes no sense here; the stack isn't contiguous with the "break", and the brk heap grows upward toward the stack anyway. Linux puts the stack very near the top of virtual address space. So of course the primary stack couldn't be allocated with (the in-kernel equivalent of) sbrk.

不，没有什么使用 MAP_GROWSDOWN，甚至没有使用辅助线程堆栈，因为它通常不能安全使用.

And no, nothing uses MAP_GROWSDOWN, not even secondary thread stacks, because it can't in general be used safely.

mmap(2) 手册页说 MAP_GROWSDOWN 是用于堆栈"的，可笑的是已经过时且具有误导性.请参阅 如何为linux 上的 clone() 系统调用?.正如 Ulrich Drepper 在 2008 年解释，使用 MAP_GROWSDOWN 的代码通常被破坏，并提议从 Linux mmap 和 glibc 头文件中删除该标志.(这显然没有发生，但在那之前 pthreads 就没有使用过它，如果有的话.)

The mmap(2) man page which says MAP_GROWSDOWN is "used for stacks" is laughably out of date and misleading. See How to mmap the stack for the clone() system call on linux?. As Ulrich Drepper explained in 2008, code using MAP_GROWSDOWN is typically broken, and proposed removing the flag from Linux mmap and from glibc headers. (This obviously didn't happen, but pthreads hasn't used it since well before then, if ever.)

MAP_GROWSDOWN 为内核内部的映射设置 VM_GROWSDOWN 标志.主线程也使用该标志来启用增长机制，因此线程堆栈可能能够以与主堆栈相同的方式增长:任意远(最多ulimit -s?) 如果堆栈指针低于页面错误位置.(Linux 不需要堆栈探测"来接触大型多页堆栈数组或 alloca 的每一页.)

MAP_GROWSDOWN sets the VM_GROWSDOWN flag for the mapping inside the kernel. The main thread also uses that flag to enable the growth mechanism, so a thread stack may be able to grow the same way the main stack does: arbitrarily far (up to ulimit -s?) if the stack pointer is below the page fault location. (Linux does not require "stack probes" to touch every page of a large multi-page stack array or alloca.)

(线程堆栈预先完全分配；只有物理页面的正常延迟分配才能支持虚拟分配，以避免浪费线程堆栈空间.)

(Thread stacks are fully allocated up front; only normal lazy allocation of physical pages to back that virtual allocation avoids wasting space for thread stacks.)

MAP_GROWSDOWN 映射也可以按照 mmap 手册页描述的方式增长:访问最低映射页面下方的保护页面"也会触发增长，即使那是在红色区域底部下方.

MAP_GROWSDOWN mapping can also grow the way the mmap man page describes: access to the "guard page" below the lowest mapped page will also trigger growth, even if that's below the bottom of the red zone.

但是主线程的堆栈具有不能使用mmap(MAP_GROWSDOWN)获得的魔力.它保留了最多ulimit -s 防止随机选择 mmap 地址创建堆栈增长的障碍.这种魔法只适用于内核程序加载器，它在 execve() 期间映射主线程的堆栈，使其不受 mmap(NULL, ...) 随机阻止未来的堆栈增长.

But the main thread's stack has magic you don't get with mmap(MAP_GROWSDOWN). It reserves the growth space up to ulimit -s to prevent random choice of mmap address from creating a roadblock to stack growth. That magic is only available to the in-kernel program-loader which maps the main thread's stack during execve(), making it safe from an mmap(NULL, ...) randomly blocking future stack growth.

mmap(MAP_FIXED) 仍然可以为主堆栈创建障碍，但如果您使用 MAP_FIXED，您 100% 负责不破坏任何东西.(无限堆栈不能超过初始如果涉及 MAP_FIXED，则为 132KiB?).MAP_FIXED 将替换现有的映射和​​保留，但其他任何东西都会将主线程的堆栈增长空间视为保留；.(我认为这是真的；值得尝试使用 MAP_FIXED_NOREPLACE 或只是一个非 NULL 提示地址)

mmap(MAP_FIXED) could still create a roadblock for the main stack, but if you use MAP_FIXED you're 100% responsible for not breaking anything. (Unlimited stack cannot grow beyond the initial 132KiB if MAP_FIXED involved?). MAP_FIXED will replace existing mappings and reservations, but anything else will treat the main thread's stack-growth space as reserved;. (I think that's true; worth trying with MAP_FIXED_NOREPLACE or just a non-NULL hint address)

见

• 堆栈内存是如何分配的使用推送"或子"x86 指令时?
• 为什么此代码在地址随机化打开时会崩溃?

pthread_create 不将 MAP_GROWSDOWN 用于线程堆栈，其他任何人也不应使用.一般不使用. Linux pthreads 默认为线程堆栈分配完整大小.这会消耗虚拟地址空间，但(直到它真正被触及)不会消耗物理页面.

pthread_create doesn't use MAP_GROWSDOWN for thread stacks, and neither should anyone else. Generally do not use. Linux pthreads by default allocates the full size for a thread stack. This costs virtual address space but (until it's actually touched) not physical pages.

为什么MAP_GROWSDOWN映射不增长的不一致结果?(有些人发现它有效，有些人发现它在触摸返回值和下面的页面时仍然出现段错误)听起来像 https://bugs.centos.org/view.php?id=4767 - MAP_GROWSDOWN 甚至可能是标准主程序之外的错误-stack VM_GROWSDOWN 映射被使用.

The inconsistent results in comments on Why is MAP_GROWSDOWN mapping does not grow? (some people finding it works, some finding it still segfaults when touching the return value and the page below) sound like https://bugs.centos.org/view.php?id=4767 - MAP_GROWSDOWN may even be buggy outside of the way the standard main-stack VM_GROWSDOWN mapping is used.

这篇关于使用 pmap 分析进程的内存映射.[堆]的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！