为什么 Linux 支持 0x7f 映射? [英] Why does Linux favor 0x7f mappings?

问题描述

通过运行一个简单的less/proc/self/maps，我看到大多数映射以55 和7F 开头.我还注意到每当我调试任何二进制文件时都会使用这些范围.

By running a simple less /proc/self/maps I see that most mappings start with 55 and 7F. I also noticed these ranges to be used whenever I debug any binary.

此外，此评论此处表明内核确实具有一些范围偏好.

In addition this comment here suggests that the kernel has indeed some range preference.

这是为什么?上述范围是否有更深层次的技术原因?如果在这些前缀之外手动mmap页面会不会有问题?

Why is that? Is there some deeper technical reason for the above ranges? Will there be a problem if I manually mmap pages outside of these prefixes?

推荐答案

首先，假设你在谈论 x86-64，我们可以看到 x86-64 的虚拟内存映射 是:

First and foremost, assuming that you are talking about x86-64, we can see that the virtual memory map for x86-64 is:

========================================================================================================================
    Start addr    |   Offset   |     End addr     |  Size   | VM area description
========================================================================================================================
                  |            |                  |         |
 0000000000000000 |    0       | 00007fffffffffff |  128 TB | user-space virtual memory, different per mm
__________________|____________|__________________|_________|___________________________________________________________
 ...              |    ...     | ...              |  ...

用户空间地址在 x86-64 中始终采用规范形式，仅使用低 48 位.见:

Userspace addresses are always in the canonical form in x86-64, using only the lower 48 bits. See:

• x86-64 规范地址?
• 地址规范形式和指针算术

这将用户空间虚拟内存的末尾置于 0x7fffffffffff.这是新程序堆栈开始的地方:即 0x7ffffffff000(减去由于 ASLR) 并增长到降低地址.

This puts the end of user-space virtual memory at 0x7fffffffffff. This is where the stack of new programs starts: that is, 0x7ffffffff000 (minus some random offset due to ASLR) and growing to lower addresses.

让我先解决一个简单的问题:

Let me address the simple question first:

如果我在这些前缀之外手动mmap页面会不会有问题?

Will there be a problem if I manually mmap pages outside of these prefixes?

完全没有，mmap 系统调用总是检查被请求的地址，它会拒绝映射与已经映射的内存区域重叠的页面或完全无效地址的页面(例如 addr 或 addr > 0x7ffffffff000).

Not at all, the mmap syscall always checks the address that is being requested, and it will refuse to map pages that overlap an already mapped memory area or pages at completely invalid addresses (e.g. addr < mmap_min_addr or addr > 0x7ffffffff000).

现在...直接进入 Linux 内核代码，正是在内核 ELF 加载器中 (fs/binfmt_elf.c:960)，我们可以看到一个很长的注释:

Now... diving straight into the Linux kernel code, precisely in the kernel ELF loader (fs/binfmt_elf.c:960), we can see a pretty long and esplicative comment:

/*
 * This logic is run once for the first LOAD Program
 * Header for ET_DYN binaries to calculate the
 * randomization (load_bias) for all the LOAD
 * Program Headers, and to calculate the entire
 * size of the ELF mapping (total_size). (Note that
 * load_addr_set is set to true later once the
 * initial mapping is performed.)
 *
 * There are effectively two types of ET_DYN
 * binaries: programs (i.e. PIE: ET_DYN with INTERP)
 * and loaders (ET_DYN without INTERP, since they
 * _are_ the ELF interpreter). The loaders must
 * be loaded away from programs since the program
 * may otherwise collide with the loader (especially
 * for ET_EXEC which does not have a randomized
 * position). For example to handle invocations of
 * "./ld.so someprog" to test out a new version of
 * the loader, the subsequent program that the
 * loader loads must avoid the loader itself, so
 * they cannot share the same load range. Sufficient
 * room for the brk must be allocated with the
 * loader as well, since brk must be available with
 * the loader.
 *
 * Therefore, programs are loaded offset from
 * ELF_ET_DYN_BASE and loaders are loaded into the
 * independently randomized mmap region (0 load_bias
 * without MAP_FIXED).
 */
if (interpreter) {
    load_bias = ELF_ET_DYN_BASE;
    if (current->flags & PF_RANDOMIZE)
        load_bias += arch_mmap_rnd();
    elf_flags |= MAP_FIXED;
} else
    load_bias = 0;

简而言之，ELF有两种类型位置独立的可执行文件:

In short, there are two types of ELF Position Independent Executables:

1. 普通程序:它们需要加载器才能运行.这基本上代表了普通 Linux 系统上 99.9% 的 ELF 程序.加载器的路径在 ELF 程序头中指定，程序头类型为 PT_INTERP.

Loaders:loader 是一个 ELF，不指定 PT_INTERP 程序头，负责加载和启动正常程序.在实际启动正在加载的程序之前，它还会在幕后做一些花哨的事情(解决重定位、加载所需的库等).

Loaders: a loader is an ELF that does not specify a PT_INTERP program header, and that is responsible for loading and starting normal programs. It also does a bunch of fancy stuff behind the scenes (resolve relocations, load needed libraries, etc.) before actually starting the program that is being loaded.

当内核通过 execve 系统调用执行新的 ELF 时，它需要将程序本身和加载器映射到内存中.然后控制将传递给加载器，加载器将解析和映射所有需要的共享库，最后将控制传递给程序.由于程序及其加载器都需要映射，内核需要确保这些映射不重叠(并且加载器未来的映射请求不会重叠).

When the kernel executes a new ELF through an execve syscall, it needs to map into memory the program itself and the loader. Control will then be passed to the loader that will resolve and map all needed shared libraries and finally pass control to the program. Since both the program and its loader need to be mapped, the kernel needs to make sure that those mappings don't overlap (and also that future mapping requests by the loader will not overlap).

为了做到这一点，加载器被映射到堆栈附近，(在比堆栈低的地址，但有一定的容忍度，因为如果需要，可以通过添加更多页面来允许堆栈增长)，留下将 ASLR 应用于 mmap 本身.然后使用 load_bias(如上面的代码片段所示)映射程序，使其离加载器足够远(位于低得多的地址).

In order to do this, the loader is mapped near the stack, (at a lower address than the stack, but with some tolerance, since the stack is allowed to grow by adding more pages if needed), leaving the duty of applying ASLR to mmap itself. The program is then mapped using a load_bias (as seen in the above snippet) to put it far enough from the loader (at a much lower address).

如果我们看看ELF_ET_DYN_BASE，我们看到它依赖于架构，并且在 x86-64 上它评估为:

If we take a look at ELF_ET_DYN_BASE, we see that it is architecture dependent and on x86-64 it evaluates to:

((1ULL << 47) - (1 << 12)) / 3 * 2 == 0x555555554aaa

基本上大约 2/3 的 TASK_SIZE.然后调整 load_bias 添加 arch_mmap_rnd() 字节，如果启用了 ASLR，最后页面对齐.归根结底，这就是为什么我们通常会看到以 0x55 开头的程序地址.

Basically around 2/3 of TASK_SIZE. That load_bias is then adjusted adding arch_mmap_rnd() bytes if ASLR is enabled, and finally page-aligned. At the end of the day, this is the reason why we usually see addresses starting with 0x55 for programs.

当控制传递给加载器时，进程的虚拟内存区域已经被定义，并且不指定地址的连续mmap系统调用将返回从加载器附近开始递减的地址.由于我们刚刚看到加载器映射到堆栈附近，并且堆栈位于用户地址空间的最末端，这就是为什么我们通常看到以 0x7f 开头的地址的原因图书馆.

When control is passed to the loader, the virtual memory area for the process has already been defined, and successive mmap syscalls that do not specify an address will return decreasing addresses starting near the loader. Since as we just saw the loader is mapped near the stack, and the stack is at the very end of the user address space, this is the reason why we usually see addresses starting with 0x7f for libraries.

上述情况有一个常见的例外.在直接调用加载器的情况下，例如:

There is a common exception to the above. In the case the loader is invoked directly, like for example:

/lib/x86_64-linux-gnu/ld-2.24.so ./myprog

在这种情况下，内核不会映射 ./mpyprog 并将其留给加载器.因此，./myprog 将被加载器映射到某个 0x7f... 地址.

The kernel will not map ./mpyprog in this case and will leave that to the loader. As a consequence, ./myprog will be mapped at some 0x7f... address by the loader.

您可能想知道:为什么内核不总是让加载程序映射程序，或者为什么程序不在加载程序之前/之后映射?对此我没有 100% 确定的答案，但我想到了几个原因:

You may be wondering: why doesn't the kernel always let the loader map the program then, or why isn't the program just mapped right before/after the loader? I don't have a 100% definitive answer for this, but a few reasons come to mind:

1. 一致性:让内核自己将 ELF 加载到内存中，而不依赖于加载器可以避免麻烦.如果不是这种情况，内核将完全依赖于用户空间加载器，这是不可取的(这也可能是部分安全问题).

1. Consistency: making the kernel itself load the ELF into memory without depending on the loader avoids trouble. If this wasn't the case, the kernel would fully depend on the userspace loader, which is not advisable at all (this may also partially be a security concern).

效率:我们确信至少需要映射可执行文件和它的加载器(不管任何链接库)，还不如节省宝贵的时间并立即执行而不是等待另一个具有关联上下文切换的系统调用.

Efficiency: we are sure that at least both the executable and its loader need to be mapped (regardless of any linked libraries), might as well save precious time and do it right away rather than wait for another syscall with associated context switch.

安全性:在默认情况下，将程序映射到与加载器和其他库不同的随机地址，在程序本身和加载的库之间提供了一种隔离".换句话说，泄漏"任何库地址都不会泄露程序在内存中的位置，反之亦然.以与加载程序和其他库的预定义偏移量映射程序将部分违背 ASLR 的目的.

Security: in the default scenario, mapping the program at a different randomized address than the loader and other libraries provides a sort of "isolation" between the program itself and the loaded libraries. In other words, "leaking" any library address won't reveal the program position in memory, and vice-versa. Mapping the program at a predefined offset from the loader and other libraries would instead partially defeat the purpose of ASLR.

在理想的安全驱动场景中，每个 mmap(即任何需要的库)也将被放置在一个独立于先前映射的随机地址，但这会显着影响性能.保持分配分组会导致更快的页表查找:参见 UnderstandingLinux 内核(第 3 版)，第 606 页:表 15-3.每个基数树高度的最高索引和最大文件大小.它还会导致更大的虚拟内存碎片，成为需要将大文件映射到内存的程序的真正问题.程序代码和库代码之间的隔离的实质部分已经完成，进一步的弊大于利.

In an ideal security-driven scenario, every single mmap (i.e. any needed library) would also be placed at a randomized address independent of previous mappings, but this would hurt performance significantly. Keeping allocations grouped results in faster page table lookups: see Understanding The Linux Kernel (3rd edition), page 606: Table 15-3. Highest index and maximum file size for each radix tree height. It would also cause much greater virtual memory fragmentation, becoming a real problem for programs that need to map large files to memory. The substantial part of isolation between program code and library code is already done, going further has more cons than pros.

易于调试:查看 RIP=0x55... 与 RIP=0x7f... 可立即帮助您确定要查找的位置(程序本身或库代码).

Ease of debugging: seeing RIP=0x55... vs RIP=0x7f... instantly helps figuring out where to look (program itself or library code).

这篇关于为什么 Linux 支持 0x7f 映射?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！