什么是gcc linux x86-64 C ++中的有效指针? [英] What is a valid pointer in gcc linux x86-64 C++?

查看：122
本文介绍了什么是gcc linux x86-64 C ++中的有效指针?的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！

问题描述

我正在使用gcc在一个名为linux x86-64的晦涩系统上编程C ++.我希望可能有一些人在使用相同的特定系统(也可能能够帮助我了解该系统上的有效指针). 我不在乎访问指针所指向的位置，只是想通过指针算法进行计算.

根据标准的3.9.2节:

对象指针类型的有效值表示内存中字节的地址(1.7)或空指针.

并根据 [expr.add]/4 :

将具有整数类型的表达式添加或减去时 从指针开始，结果具有指针操作数的类型.如果 表达式P指向具有n的数组对象x的元素x [i] 元素，表达式P + J和J + P(其中J的值为j) 如果0≤i + j≤则指向(可能是假设的)元素x [i + j] n; 否则，行为是不确定的.同样，表达式P- 如果0≤i-j，则J指向(可能是假想的)元素x [i-j] ≤n;否则，行为是不确定的.

根据关于有效C ++指针的 stackoverflow问题:

0x1是系统上的有效内存地址吗?好吧，对于某些嵌入式系统而言.对于大多数使用虚拟内存的操作系统，从零开始的页面被保留为无效.

好吧，这很清楚！因此，除了NULL之外，有效指针是内存中的一个字节，不，等待，它是一个数组元素，包括紧接该数组之后的元素，不，等待，这是虚拟内存页面，不，等待，这是超人！ /p>

(我想这里的超人"是指垃圾收集器" ...不是我在任何地方都读过，只是闻到了气味.严重的是，最好的垃圾收集器不会严重损坏您周围有伪造的指针；最糟糕的是，它们不时不时收集一些死对象.似乎没有什么值得搞砸指针算法的.)

因此，基本上，适当的编译器必须支持上述所有类型的有效指针.我的意思是，一个假设编译器仅凭指针 calculation 不好就具有生成未定义行为的胆量，至少可以避开上面的3个项目符号，对吗? (好的，语言律师，那是你的).

此外，对于编译器而言，其中许多定义几乎是不可能的.创建有效的内存字节的方法有很多种(想想看，懒惰的段错误陷阱微码，向我要访问数组的一部分的自定义页表系统的边带提示，...)，映射页面，或简单地创建数组.

例如，我创建一个自己的大型数组，然后让默认内存管理器在其中创建一个较小的数组:

#include <iostream>
#include <inttypes.h>
#include <assert.h>
using namespace std;

extern const char largish[1000000000000000000L];
asm("largish = 0");

int main()
{
  char* smallish = new char[1000000000];
  cout << "largish base = " << (long)largish << "\n"
       << "largish length = " << sizeof(largish) << "\n"
       << "smallish base = " << (long)smallish << "\n";
}

结果:

largish base = 0
largish length = 1000000000000000000
smallish base = 23173885579280

(不要问我如何知道，默认的内存管理器会在另一个数组中分配一些东西.这是一个晦涩的系统设置.关键是我经历了数周的调试折磨，以使此示例的工作，只是为了向您证明不同的分配技术可能相互遗忘).

鉴于Linux x86-64中支持的内存管理和组合程序模块的方式数量众多，C ++编译器实际上不知道所有数组和页面映射的各种样式

最后，为什么我要专门提及gcc?因为通常将 any 指针视为有效的指针...例如，

char* super_tricky_add_operation(char* a, long b) {return a + b;}

在阅读了所有语言规范后，您可能会期望super_tricky_add_operation(a, b)的实现充斥着未定义的行为，但实际上，这很无聊，只是一个add或lea指令.太好了，因为如果没有人撒谎，我可以将其用于非常方便和实用的事情，例如基于非零的数组用我的add指令只是为了指出无效的指针.我爱 gcc.

总而言之，似乎任何在Linux x86-64上支持标准链接工具的C ++编译器几乎都必须将 any 指针视为有效指针，而gcc似乎是其中的成员.那个俱乐部.但是我不确定100％(就是说，有足够的分数精度).

那么...谁能举一个可靠的例子来说明gcc linux x86-64中的无效指针?所谓固体，是指导致未定义的行为.并解释一下是什么引起了语言规范所允许的未定义行为?

(或提供gcc文档证明相反:所有指针均有效).

解决方案

通常，指针数学可以完全满足您的期望，而不管指针是否指向对象.

UB并不意味着它有失败.只是允许使整个程序的其余部分以某种方式表现出奇怪的行为. UB并不意味着仅指针比较结果可能是错误的"，它意味着整个程序的整个行为是不确定的.依赖违背假设的优化往往会发生这种情况.

有趣的极端情况是在虚拟地址空间的顶部包含一个数组:指向最后一句的指针将换为零，所以start < end将为假?！但是指针比较不必处理这种情况，因为Linux内核永远不会映射首页，因此指向它的指针不能指向或仅指向过去的对象.参见为什么不能在64位内核的32位Linux进程中映射(MAP_FIXED)最高的虚拟页面?

相关:

GCC 确实的最大对象大小为PTRDIFF_MAX(这是带符号的类型).因此，例如，在32位x86上，虽然您可以mmap一个，但并非所有代码生成实例都完全支持大于2GB的数组.

在中查看我的评论最大尺寸是多少在C中的数组?-对于比char宽的类型(其中C减法的结果是对象)的类型，此限制使gcc可以进行指针减法(以获取大小)而不会保留高位的进位，而不是字节，因此在asm中为(a - b) / sizeof(T).

不要问我怎么知道默认的内存管理器会在另一个数组中分配一些东西.这是一个晦涩的系统设置.关键是我经历了数周的调试折磨，以使该示例正常工作，只是向您证明了不同的分配技术可以相互忽略).

首先，您实际上从未分配用于large[]的空间.您使用嵌入式asm使其从地址0开始，但实际上没有进行映射的任何操作.

当new使用brk或mmap从内核获取新内存时，内核不会重叠现有的映射页面，因此，静态分配和动态分配实际上不会重叠.

第二，char[1000000000000000000L]〜= 2 ^ 59字节.当前的x86-64硬件和软件仅支持规范的48位虚拟地址(符号扩展为64位).这将随着下一代英特尔硬件的出现而改变，后者将添加更高级别的页表，使我们最多可使用48 + 9 = 57位地址. (仍然使用内核使用的上半部分，中间使用一个大洞.)

您的从0到〜2 ^ 59的未分配空间覆盖了x86-64 Linux上可能的所有用户空间虚拟内存地址，因此，您分配的任何内容(包括其他静态数组)都将位于此伪造的内部"数组.

从声明中删除extern const(因此实际上分配了数组 ，

(有趣的是，对于PIE可执行文件和非PIE可执行文件，我们会获得不同的错误代码，但仍在execve()尚未完成之前.)

用asm("largish = 0");欺骗编译器+链接器+运行时不是很有趣，并且会产生明显的未定义行为.

有趣的事实2:x64 MSVC不支持大于2 ^ 31-1字节的静态对象. IDK(如果它具有-mcmodel=medium等效项).基本上，GCC 不会警告所选存储模型太大的对象.

<source>(7): error C2148: total size of array must not exceed 0x7fffffff bytes

<source>(13): warning C4311: 'type cast': pointer truncation from 'char *' to 'long'
<source>(14): error C2070: 'char [-1486618624]': illegal sizeof operand
<source>(15): warning C4311: 'type cast': pointer truncation from 'char *' to 'long'

此外，它指出long通常是错误的指针类型(因为Windows x64是LLP64 ABI，其中long是32位).您需要intptr_t或uintptr_t，或者等价于printf("%p")的东西，它会打印原始的void*.

I am programming C++ using gcc on an obscure system called linux x86-64. I was hoping that may be there are a few folks out there who have used this same, specific system (and might also be able to help me understand what is a valid pointer on this system). I do not care to access the location pointed to by the pointer, just want to calculate it via pointer arithmetic.

According to section 3.9.2 of the standard:

A valid value of an object pointer type represents either the address of a byte in memory (1.7) or a null pointer.

And according to [expr.add]/4:

When an expression that has integral type is added to or subtracted from a pointer, the result has the type of the pointer operand. If the expression P points to element x[i] of an array object x with n elements, the expressions P + J and J + P (where J has the value j) point to the (possibly-hypothetical) element x[i + j] if 0 ≤ i + j ≤ n; otherwise, the behavior is undefined. Likewise, the expression P - J points to the (possibly-hypothetical) element x[i − j] if 0 ≤ i − j ≤ n; otherwise, the behavior is undefined.

And according to a stackoverflow question on valid C++ pointers in general:

Is 0x1 a valid memory address on your system? Well, for some embedded systems it is. For most OSes using virtual memory, the page beginning at zero is reserved as invalid.

Well, that makes it perfectly clear! So, besides NULL, a valid pointer is a byte in memory, no, wait, it's an array element including the element right after the array, no, wait, it's a virtual memory page, no, wait, it's Superman!

(I guess that by "Superman" here I mean "garbage collectors"... not that I read that anywhere, just smelled it. Seriously, though, all the best garbage collectors don't break in a serious way if you have bogus pointers lying around; at worst they just don't collect a few dead objects every now and then. Doesn't seem like anything worth messing up pointer arithmetic for.).

So, basically, a proper compiler would have to support all of the above flavors of valid pointers. I mean, a hypothetical compiler having the audacity to generate undefined behavior just because a pointer calculation is bad would be dodging at least the 3 bullets above, right? (OK, language lawyers, that one's yours).

Furthermore, many of these definitions are next to impossible for a compiler to know about. There are just so many ways of creating a valid memory byte (think lazy segfault trap microcode, sideband hints to a custom pagetable system that I'm about to access part of an array, ...), mapping a page, or simply creating an array.

Take, for example, a largish array I created myself, and a smallish array that I let the default memory manager create inside of that:

#include <iostream>
#include <inttypes.h>
#include <assert.h>
using namespace std;

extern const char largish[1000000000000000000L];
asm("largish = 0");

int main()
{
  char* smallish = new char[1000000000];
  cout << "largish base = " << (long)largish << "\n"
       << "largish length = " << sizeof(largish) << "\n"
       << "smallish base = " << (long)smallish << "\n";
}

Result:

largish base = 0
largish length = 1000000000000000000
smallish base = 23173885579280

(Don't ask how I knew that the default memory manager would allocate something inside of the other array. It's an obscure system setting. The point is I went through weeks of debugging torment to make this example work, just to prove to you that different allocation techniques can be oblivious to one another).

Given the number of ways of managing memory and combining program modules that are supported in linux x86-64, a C++ compiler really can't know about all of the arrays and various styles of page mappings.

Finally, why do I mention gcc specifically? Because it often seems to treat any pointer as a valid pointer... Take, for instance:

char* super_tricky_add_operation(char* a, long b) {return a + b;}

While after reading all the language specs you might expect the implementation of super_tricky_add_operation(a, b) to be rife with undefined behavior, it is in fact very boring, just an add or lea instruction. Which is so great, because I can use it for very convenient and practical things like non-zero-based arrays if nobody is putzing with my add instructions just to make a point about invalid pointers. I love gcc.

In summary, it seems that any C++ compiler supporting standard linkage tools on linux x86-64 would almost have to treat any pointer as a valid pointer, and gcc appears to be a member of that club. But I'm not quite 100% sure (given enough fractional precision, that is).

So... can anyone give a solid example of an invalid pointer in gcc linux x86-64? By solid I mean leading to undefined behavior. And explain what gives rise to the undefined behavior allowed by the language specs?

(or provide gcc documentation proving the contrary: that all pointers are valid).

解决方案

Usually pointer math does exactly what you'd expect regardless of whether pointers are pointing at objects or not.

UB doesn't mean it has to fail. Only that it's allowed to make the whole rest of the program behave strangely in some way. UB doesn't mean that just the pointer-compare result can be "wrong", it means the entire behaviour of the whole program is undefined. This tends to happen with optimizations that depend on a violated assumption.

Interesting corner cases include an array at the very top of virtual address space: a pointer to one-past-the-end would wrap to zero, so start < end would be false?!? But pointer comparison doesn't have to handle that case, because the Linux kernel won't ever map the top page, so pointers into it can't be pointing into or just past objects. See Why can't I mmap(MAP_FIXED) the highest virtual page in a 32-bit Linux process on a 64-bit kernel?

Related:

GCC does have a max object size of PTRDIFF_MAX (which is a signed type). So for example, on 32-bit x86, an array larger than 2GB isn't fully supported for all cases of code-gen, although you can mmap one.

See my comment on What is the maximum size of an array in C? - this restriction lets gcc implement pointer subtraction (to get a size) without keeping the carry-out from the high bit, for types wider than char where the C subtraction result is in objects, not bytes, so in asm it's (a - b) / sizeof(T).

Don't ask how I knew that the default memory manager would allocate something inside of the other array. It's an obscure system setting. The point is I went through weeks of debugging torment to make this example work, just to prove to you that different allocation techniques can be oblivious to one another).

First of all, you never actually allocated the space for large[]. You used inline asm to make it start at address 0, but did nothing to actually get those pages mapped.

The kernel won't overlap existing mapped pages when new uses brk or mmap to get new memory from the kernel, so in fact static and dynamic allocation can't overlap.

Second, char[1000000000000000000L] ~= 2^59 bytes. Current x86-64 hardware and software only support canonical 48-bit virtual addresses (sign-extended to 64-bit). This will change with a future generation of Intel hardware which adds another level of page tables, taking us up to 48+9 = 57-bit addresses. (Still with the top half used by the kernel, and a big hole in the middle.)

Your unallocated space from 0 to ~2^59 covers all user-space virtual memory addresses that are possible on x86-64 Linux, so of course anything you allocate (including other static arrays) will be somewhere "inside" this fake array.

Removing the extern const from the declaration (so the array is actually allocated, https://godbolt.org/z/Hp2Exc) runs into the following problems:

//extern const 
char largish[1000000000000000000L];
//asm("largish = 0");

/* rest of the code unchanged */

  • RIP-relative or 32-bit absolute (-fno-pie -no-pie) addressing can't reach static data that gets linked after large[] in the BSS, with the default code model (-mcmodel=small where all static code+data is assumed to fit in 2GB)

    $ g++ -O2 large.cpp
/usr/bin/ld: /tmp/cc876exP.o: in function `_GLOBAL__sub_I_largish':
large.cpp:(.text.startup+0xd7): relocation truncated to fit: R_X86_64_PC32 against `.bss'
/usr/bin/ld: large.cpp:(.text.startup+0xf5): relocation truncated to fit: R_X86_64_PC32 against `.bss'
collect2: error: ld returned 1 exit status

  • compiling with -mcmodel=medium places large[] in a large-data section where it doesn't interfere with addressing other static data, but it itself is addressed using 64-bit absolute addressing. (Or -mcmodel=large does that for all static code/data, so every call is indirect movabs reg,imm64 / call reg instead of call rel32.)

    That lets us compile and link, but then the executable won't run because the kernel knows that only 48-bit virtual addresses are supported and won't map the program in its ELF loader before running it, or for PIE before running ld.so on it.

    peter@volta:/tmp$ g++ -fno-pie -no-pie -mcmodel=medium -O2 large.cpp
peter@volta:/tmp$ strace ./a.out 
execve("./a.out", ["./a.out"], 0x7ffd788a4b60 /* 52 vars */) = -1 EINVAL (Invalid argument)
+++ killed by SIGSEGV +++
Segmentation fault (core dumped)
peter@volta:/tmp$ g++ -mcmodel=medium -O2 large.cpp
peter@volta:/tmp$ strace ./a.out 
execve("./a.out", ["./a.out"], 0x7ffdd3bbad00 /* 52 vars */) = -1 ENOMEM (Cannot allocate memory)
+++ killed by SIGSEGV +++
Segmentation fault (core dumped)

(Interesting that we get different error codes for PIE vs non-PIE executables, but still before execve() even completes.)

Tricking the compiler + linker + runtime with asm("largish = 0"); is not very interesting, and creates obvious undefined behaviour.

Fun fact #2: x64 MSVC doesn't support static objects larger than 2^31-1 bytes. IDK if it has a -mcmodel=medium equivalent. Basically GCC fails to warn about objects too large for the selected memory model.

<source>(7): error C2148: total size of array must not exceed 0x7fffffff bytes

<source>(13): warning C4311: 'type cast': pointer truncation from 'char *' to 'long'
<source>(14): error C2070: 'char [-1486618624]': illegal sizeof operand
<source>(15): warning C4311: 'type cast': pointer truncation from 'char *' to 'long'

Also, it points out that long is the wrong type for pointers in general (because Windows x64 is an LLP64 ABI, where long is 32 bits). You want intptr_t or uintptr_t, or something equivalent to printf("%p") that prints a raw void*.

这篇关于什么是gcc linux x86-64 C ++中的有效指针?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！

查看全文
相关文章
C/C++开发最新文章
热门教程
热门工具
登录 关闭
扫码关注1秒登录
发送“验证码”获取 | 15天全站免登陆