允许结构字段溢出到下一个字段 [英] Allowing struct field to overflow to the next field

问题描述

考虑以下简单示例:

  struct __attribute__((__packed__)){整数代码[1];int place_holder [100];} s;无效测试(int n){我对于(i = 0; i< n; i ++){s.code [i] = 1;}} 

for循环正在写入大小为1的字段 code . code 之后的下一个字段是 place_holder .
我希望在 n>的情况下，1 ，则对 code 数组的写操作将溢出，并且将 1 写入 place_holder .

但是，当使用 -O2 进行编译时(在gcc 4.9.4上，但也可能在其他版本上)，会发生一些有趣的事情.
编译器确定代码可能溢出数组 code ，并且将循环展开限制为1次迭代.

很容易看出，在使用 -fdump-tree-all 进行编译并查看最后的树传递("t.optimized")时:

 ;;功能测试(test，funcdef_no = 0，decl_uid = 1366，symbol_order = 1)卸下基本块5测试(int n){< bb 2> ;:#DEBUG i =>0#DEBUG i =>0如果(n_4(D)> 0)转到< bb 3> ;;别的转到< bb 4> ;;< bb 3> ;:s.code [0] = 1;#DEBUG i =>1个#DEBUG i =>1个< bb 4> ;:返回;} 

因此，在这种情况下，编译器将循环完全展开为单个迭代.

我的问题是:

1. 从C规范的角度来看，是否有意从一个结构成员溢出到另一个结构成员是非法或未定义的行为?
   假设我知道内存中的结构布局，并且故意使 code 数组溢出时正在做什么.
2. 有没有办法防止gcc在这种情况下展开循环?我知道我可以完全防止循环展开，但是我仍然对其他情况下的循环展开感兴趣.我还怀疑编译器所做的分析可能会影响循环展开以外的传递.
   gcc假设我访问数组时不会溢出，所以我真正要寻找的是告诉编译器不要采取这种假设的方式(通过提供一些编译器选项).

我知道编写这样的代码会从一个字段溢出到另一个字段是一种不好的做法，而且我也不想编写这样的代码.
我也知道将数组(可能为零大小)作为 last 结构字段以允许其溢出的做法，这是编译器很好地支持的，而在这种情况下，数组 code 不是最后一个字段.
因此，这不是如何修复代码"的问题，而是理解编译器假设并影响它们的问题.

当我观察已经以这种方式编写的现有代码，并对其进行调试以找出其行为不如原始开发人员期望的那样时，就会出现这些问题.
风险是代码中还有其他地方存在此类问题.静态分析工具可以帮助您找出问题，但我也想知道是否有一种方法可以使编译器容忍此类代码并仍然生成我们期望的结果.

更新

我对上面的问题(1)有明确的答案，但对问题(2)却没有.

• gcc是否可以通过某些编译选项将其作为扩展?
• 是否有办法至少在gcc识别警告时得到警告?(并且通过优化将其清楚地标识出来).
  为了在大型现有代码库中识别此类情况，这一点很重要.

解决方案

从C规范的角度来看，是否有意从一个结构成员溢出到另一个结构成员是非法或未定义的行为?

这是未定义的行为. arr [i] 运算符是围绕 *(arr + i)的语法糖.因此，数组访问从§7和§8归结为指针算术的二进制 + 运算符C17 6.5.6加法运算符:

出于这些运算符的目的，指向对象的指针不是对象的元素数组的行为与指向长度为1的数组的第一个元素的指针相同，对象的类型作为其元素类型.

将具有整数类型的表达式添加到指针或从指针中减去时，结果具有指针操作数的类型./-/
如果两个指针都操作数和结果指向同一数组对象的元素，或者指向最后一个数组的元素数组对象的元素，求值不应产生溢出；否则，行为是不确定的.如果结果指向数组对象的最后一个元素之后，它将不得用作被评估的一元*运算符的操作数.

您已经注意到，优化编译器可能会利用这些规则来生成更快的代码.

---

在这种情况下，有没有办法防止gcc展开循环?

有一个可以使用的特殊例外规则C17 6.3.2.3/7:

将指向对象的指针转换为指向字符类型的指针时，结果指向对象的最低寻址字节.的连续增量结果，直到对象的大小，才产生指向对象剩余字节的指针.

此外，由于C17 6.5§7中的另一条特殊规则，严格的别名也不适用于字符类型

一个对象的存储值只能由具有以下之一的左值表达式访问:以下类型:...字符类型.

这两个特殊规则和谐共处.因此，假设我们在指针转换过程中不会弄乱对齐等，这意味着我们可以这样做:

  unsigned char * i;对于(i =(unsigned char *)& mystruct; i<(unsigned char *)(& mystruct + 1); i ++){do_something(* i);} 

然而，这可以读取填充字节等，因此它是实现定义的".但是从理论上讲，您可以访问每个字节的结构字节，并且只要以逐字节为单位计算结构偏移量，就可以以这种方式遍历结构的多个成员(或任何其他对象).

---

据我所知，这个看起来很可疑的代码应该定义清楚:

  #include< stdint.h>#include< string.h>#include< stdio.h>结构__attribute__((__packed__)){整数代码[1];int place_holder [100];} s;无效测试(int val，int n){for(unsigned char * i =(unsigned char *)&s;我<(unsigned char *)&s + n * sizeof(int);我+ = _Alignof(int)){if((uintptr_t)i％_Alignof(int)== 0)//并非真正必要，只是防御程序.{memcpy(i，& val，sizeof(int));printf(将％d写入地址％p \ n"，val，(void *)i)；}}}int main(无效){测试(42，3);printf(％d％d％d \ n"，s.code [0]，s.place_holder [0]，s.place_holder [1])；} 

这在gcc和clang(x86)上正常工作.它有多有效，这又是另外一回事了.不过，请不要编写这样的代码.

Consider the following simple example:

struct __attribute__ ((__packed__)) {
 int code[1];
 int place_holder[100];
} s;

void test(int n)
{
 int i;

 for (i = 0; i < n; i++) {
  s.code[i] = 1;
 }
}

The for-loop is writing to the field code, which is of size 1. The next field after code is place_holder.
I would expect that in case of n > 1, the write to code array would overflow and 1 would be written to place_holder.

However, when compiling with -O2 (on gcc 4.9.4 but probably on other versions as well) something interesting happens.
The compiler identifies that the code might overflow array code, and limits loop unrolling to 1 iteration.

It's easy to see that when compiling with -fdump-tree-all and looking at the last tree pass ("t.optimized"):

;; Function test (test, funcdef_no=0, decl_uid=1366, symbol_order=1)

Removing basic block 5
test (int n)
{
  <bb 2>:
  # DEBUG i => 0
  # DEBUG i => 0
  if (n_4(D) > 0)
    goto <bb 3>;
  else
    goto <bb 4>;

  <bb 3>:
  s.code[0] = 1;
  # DEBUG i => 1
  # DEBUG i => 1

  <bb 4>:
  return;

}

So in this case the compiler completely unrolled the loop to a single iteration.

My questions are:

1. From C specification viewpoint, is overflowing (deliberately) from one struct member to the next is illegal or undefined behavior?
   Let's assume I'm aware of the struct layout in memory and know what I'm doing when deliberately overflowing the code array.
2. Is there a way to prevent gcc from unrolling the loop in such case? I know I can completely prevent loop unrolling, however I'm still interested in loop unrolling on other cases. I also suspect that the analysis the compiler is doing might affect passes other than loop unrolling.
   gcc is assuming I'm not going to overflow when accessing my array, so what I'm really looking for is way to tell the compiler not to take this assumption (by providing some compiler option).

I'm aware it's a bad practice to write such code that overflows from one field to another, and I'm not intending to write such code.
I'm also aware of the practice to put an array (possibly zero sized) as the last struct field to allow it to overflow, this is well supported by compilers, while in this case the array code is not the last field.
So this is not a question of "how to fix the code", but rather a question of understanding the compiler assumptions and affecting them.

These questions came up when I observed existing code that was already written in such way, and debugged it to find out why it's not behaving as the original developer expected it to behave.
The risk is that there are other places in the code where such problem exists. Static analysis tools can help to find out, but I would also like to know if there's a way to make the compiler tolerate such code and still generate the result we would expect.

Update

I got clear answer to question (1) above, but not for question (2).

• Can gcc allow this as an extension, by some compile options?
• Is there a way to at least get a warning when gcc identifies it? (and it clearly identifies it, by optimizing things out).
  That's important in order to identify such cases in a large existing code base.

解决方案

From C specification viewpoint, is overflowing (deliberately) from one struct member to the next is illegal or undefined behavior?

It is undefined behavior. The arr[i] operator is syntactic sugar around *(arr + i). So array access boils down to the binary + operator for pointer arithmetic, C17 6.5.6 additive operators, from §7 and §8:

For the purposes of these operators, a pointer to an object that is not an element of an array behaves the same as a pointer to the first element of an array of length one with the type of the object as its element type.

When an expression that has integer type is added to or subtracted from a pointer, the result has the type of the pointer operand. /--/
If both the pointer operand and the result point to elements of the same array object, or one past the last element of the array object, the evaluation shall not produce an overflow; otherwise, the behavior is undefined. If the result points one past the last element of the array object, it shall not be used as the operand of a unary * operator that is evaluated.

As you noticed, optimizing compilers might exploit these rules to produce faster code.

---

Is there a way to prevent gcc from unrolling the loop in such case?

There is a a special exception rule that can be used, C17 6.3.2.3/7:

When a pointer to an object is converted to a pointer to a character type, the result points to the lowest addressed byte of the object. Successive increments of the result, up to the size of the object, yield pointers to the remaining bytes of the object.

Also, strict aliasing does not apply to character types, because of another special rule in C17 6.5 §7

An object shall have its stored value accessed only by an lvalue expression that has one of the following types: ... a character type.

These two special rules co-exist in harmony. So assuming we don't mess up alignment etc during the pointer conversion, this means that we are allowed to do this:

unsigned char* i;
for(i = (unsigned char*)&mystruct; i < (unsigned char*)(&mystruct + 1); i++)
{
  do_something(*i);
}

This may however read padding bytes etc so it's "implementation-defined". But in theory you can access the struct byte per byte, and as long as the struct offsets are calculated on byte-per-byte basis, you can iterate across multiple members of the struct (or any other object) in this manner.

---

As far as I can tell, this very questionable-looking code should be well-defined:

#include <stdint.h>
#include <string.h>
#include <stdio.h>

struct __attribute__ ((__packed__)) {
 int code[1];
 int place_holder[100];
} s;

void test(int val, int n)
{
  for (unsigned char* i = (unsigned char*)&s; 
       i < (unsigned char*)&s + n*sizeof(int); 
       i += _Alignof(int)) 
  {
    if((uintptr_t)i % _Alignof(int) == 0) // not really necessary, just defensive prog.
    {
      memcpy(i, &val, sizeof(int));
      printf("Writing %d to address %p\n", val, (void*)i);
    }
  }
}

int main (void)
{
  test(42, 3);
  printf("%d %d %d\n", s.code[0], s.place_holder[0], s.place_holder[1]);
}

This works fine on gcc and clang (x86). How efficient it is, well that's another story. Please don't write code like this, though.

这篇关于允许结构字段溢出到下一个字段的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！