__attribute__((packed)) 对嵌套结构数组的影响? [英] Effects of __attribute__((packed)) on nested array of structures?

问题描述

问题

我正在努力通过网络将原始结构发送到另一端的已知程序，但不得不担心用于对齐结构的静默引入的内存(包括其他问题，如字节顺序).我正在使用的是这样的:

typedef struct __attribute__((packed)){uint16_t 字段1；uint16_t 字段2；uint16_t 字段3；} 打包_1_s;typedef struct __attribute__((packed)){uint16_t 字段A；uint16_t 字段B；packed_1_s structArray[10];} 打包_2_s;typedef struct __attribute__((packed)){uint16_t 字段X；packed_2_s 字段Y；uint8_t 数组Z[20]；} 数据_s;

我了解通常 packed_1_s 结构可以/将为结构的每个实例分配额外的空间，以将其填充到编译器的首选大小(取决于为其构建的硬件)，并且首选大小可以是任何地方从 2 个字节到 64 个字节(最近).通常，如果我在 packed_2_s 中有一个 packed_1_s 实例，则不会有任何问题，但是当您尝试将元素放入数组时，我会理解存在一些差异.

尝试的解决方案

gcc 文档似乎表明，通过在 packed_2_s 定义中简单地包含 packed 属性，这些字段，即使它们是数组，都将尽可能紧密地打包，并且不会向 packed_2_s 结构添加空间以对齐数组的元素.尽管关于 align() 属性的文档表明，数组的处理方式与其他字段不同，并且需要直接在字段上设置 align/packed 属性才能修改添加的额外间距以匹配指定的对齐方式(或缺少对齐方式).我尝试在 structArray 字段上设置打包属性，当这不起作用时，通过在上面的代码中设置 arrayZ 上的打包属性进行测试:

packed_1_s structArray[10] __attribute__((packed));uint8_t arrayZ[20] __attribute__((packed));

两次尝试都给了我一个编译器警告，提示在此上下文中不理解打包属性并将被跳过(我使用-Wall"构建的好东西).

我希望解决这个问题的方法是使用属性 align(1)，表示所需的 1 字节对齐方式与打包属性相当，但文档说 align() 属性只能 增加对齐和packed应该用于减少对齐.

注意事项

从我可以从 GCC 文档中确定的情况来看，似乎有 3 种主要情况下插入了额外的内存.

<块引用>

1. 结构内容可能有额外的内存分配给结构本身以更改字段之间的间距.
   有效地定义结构的内存映射结构中的内容可能会改变(虽然不是顺序元素).
2. 结构可能有额外的内存分配给它们来填充它们的整体尺寸更有效.这一般是打算让其他变量在声明其中之一之后出现它们的实例与结构实例，其中块"由系统/编译器定义.
3. 数组，无论是否在一个结构中，都可能有额外的添加到它们的内存以将元素转换为有效的对齐方式.

据我所知，packed 属性可用于影响结构并阻止在上述情况 1 和 2 中添加的额外内存，但似乎没有办法在我的情况下处理上述情况 3编译器.

问题

有什么方法可以保证 data_s 结构绝对没有额外的空间添加到它或它的任何子结构中，所以我在内存映射中没有编译器相关的变化?我是否误解了编译器可以插入空间来故意移动内存映射的情况?

编辑

我与当地的大师讨论了一些问题，听起来我对上面的案例 3 有一些误解.数组中的元素之间没有插入空格，但是保证它们正确对齐的额外空间被添加到结构本身.显然，这表明诸如sizeof(structureOnlyContaining_uint32_t)"之类的东西并不总是返回4"，因为可能会添加额外的空间来对齐正在使用的编译器上的 uint32_t 数据类型.结果真的只有2种情况:

<块引用>

1. 结构的内存映射中的字段之间的偏移量更大.
   可以修改字段之间的空间以对齐每个字段.这可以使用 packed 或 align() 属性进行更改.
2. 结构末端填充.结构的大小，由返回sizeof()，可以修改，因此结构的数组最终为系统正确对齐.这允许所有系统假设结构的开头将始终对齐，从而导致问题如果他们不是.这似乎不受 pack 或 align 属性的影响.

由于新的情况 2，结构中的数组元素不一定遵守结构上指定的打包或 align() 属性，尽管数组的开头和紧随数组的字段会这样做.

然后我的问题是关于如何处理 packed_2_s 中的 structArray，因为整个数组的大小不能完全由 packed 属性来保证.有没有办法保证整个 structArray 字段的大小是固定的?应该注意的是，我不能过多地增加 packed_1_s 的大小，因为 data_s 结构需要保持尽可能小(它在流式传输场景中替换音频/视频数据).

解决方案

__attribute__((packed))注意以下几点:

• 当 packed 用于结构声明时，它会压缩其字段，例如 sizeof(structure) == sizeof(first_member) + ... + sizeof(last_member).

• 这里，数组只是结构的一个成员.打包数组的包含结构不会改变数组的大小.事实上，(任何)数组的大小总是 sizeof(element) * number_of_elements.

• 同样，包装内部结构的包含结构不会改变内部结构的大小.结构的大小完全由它的声明决定，无论你在哪里使用都是一样的.

• 打包一个结构会使其需要对齐一个字节(即它可以放在内存中的任何位置).

• 在访问打包结构的字段时，打包会引入对齐问题.编译器会在直接访问字段时考虑这一点，但不会在通过指针访问时考虑.当然，这不适用于具有所需对齐方式的字段(例如 char 或其他打包结构).请参阅我对类似问题的回答，其中包含一个演示问题的程序通过指针访问成员.

最后，回答这个问题，

<块引用>

有什么办法可以保证data_s结构会有绝对没有额外的空间添加到它或它的任何子结构，所以我在内存中没有编译器依赖的变化地图?

是的.以递归方式将结构及其包含的所有结构声明为已打包.

还要注意，packed 属性适用于结构声明，而不适用于类型.没有被声明为非打包的结构的打包版本.当您在某处使用结构时，当且仅当结构本身被声明为打包时，它(其成员)才会被打包.结构的大小完全由其声明决定.

更新:出于某种原因，您仍然对数组感到困惑.我提供的解决方案(声明所有结构已打包)也适用于数组.例如:

struct elem_struct {uint32_t x;} __attribute__((packed));//打包保证 sizeof(struct elem_struct) = sizeof(uint32_t) = 4结构数组结构{结构 elem_struct arr[10];} __attribute__((packed));//打包保证 sizeof(struct array_struct) =//= sizeof(struct elem_struct[10]) = 10 * sizeof(struct elem_struct)//= 10 * 4 = 40

关于数组的另外两点是正确的——但只有在结构没有被打包的情况下.打包强制结构的字段是连续的，这确实会产生对齐问题，如果不使用打包，可以通过在成员之间插入空白空间并填充结构来解决(参见第 I 点已经提出了对齐).

The Problem

I'm working on sending a raw structure over a network to a known program on the other side, but have to worry about the silently introduced memory used for aligning the structures (other issues like endianness are covered). What I'm working with is something like:

typedef struct __attribute__((packed))
{
   uint16_t field1;
   uint16_t field2;
   uint16_t field3;
} packed_1_s;

typedef struct __attribute__((packed))
{
   uint16_t fieldA;
   uint16_t fieldB;
   packed_1_s structArray[10];
} packed_2_s;

typedef struct __attribute__((packed))
{
   uint16_t fieldX;
   packed_2_s fieldY;
   uint8_t arrayZ[20];
} data_s;

I understand that normally the packed_1_s structure could/would have additional space allocated for every instance of the structure to fill it out to the compiler's favored size (dependent on the hardware it's being built for), and that favored size can be anywhere from 2 bytes to 64 bytes (most recently). Normally if I had a single instance of packed_1_s in packed_2_s there would be no problem, but I'm given to understand there's some differences when you try to put elements in an array.

Attempted Solutions

The gcc documentation seems to suggest that by simply including the packed attribute in the packed_2_s definition, the fields, even if they're arrays, will all be as tightly packed as possible and won't add space to the packed_2_s structure to align the elements of the array. The documentation on the align() attribute though suggests that arrays are handled differently than other fields and need the align/packed attribute set on the field directly for it to modify the extra spacing added to match the specified alignment (or lack thereof). I tried setting the packed attribute on both the structArray field, and when that didn't work, did a test by setting the packed attribute on arrayZ in the code above:

packed_1_s structArray[10] __attribute__((packed));

uint8_t arrayZ[20] __attribute__((packed));

Both attempts gave me a compiler warning that the packed attribute wasn't understood in this context and would be skipped (good thing I build with "-Wall").

I hoped a way to get around the issue would be to use attribute align(1), indicating a desired alignment of 1 byte which is comparable to the packed attribute, but documentation says the align() attribute can only increase the alignment and packed should be used to decrease the alignment.

Considerations

From what I can determine from the GCC documentation, it seems like there's 3 major cases of additional memory being inserted.

1. Structure contents may have additional memory allocated to the structure itself to change the spacing between fields.
   Effectively, the definition of the memory map of the structure contents within the structure may change (though not the order of the elements).
2. Structures may have additional memory allocated to them to fill them out to a more efficient overall size. This is generally intended so other variables coming after a declaration of one of their instances doesn't fall within the same "block" as the structure instance where a "block" is defined by the system/compiler.
3. Arrays, whether within a structure or not, may have additional memory added to them to shift the elements to an efficient alignment.

As far as I can tell, the packed attribute can be used to affect the structures and block the additional memory added in case 1 and 2 above, but there doesn't seem to be a way to handle case 3 above on my compiler(s).

The Question

Is there any way to guarantee that the data_s structure will have absolutely no additional space added to it or any of its sub-structures so I don't have compiler dependent shifts in the memory map? Am I misunderstanding the cases where the compiler can insert space to intentionally shift the memory map?

EDIT

I discussed some of the issues with my local guru and it sounds like I have some misunderstanding of case 3 above. The elements in the array don't have space inserted between them, but the additional space to guarantee they align correctly is added to the structure itself. Apparently this suggests things like "sizeof(structureOnlyContaining_uint32_t)" won't always return "4" since additional space may be added to align the uint32_t datatype on the compiler being used. The result is that there are really only 2 cases:

1. Larger offsets in between fields in the memory map of the structure.
   The space between the fields can be modified to align each field. This can be changed using the packed or align() attributes.
2. Structure end padding. The size for a structure, as returned by sizeof(), can be modified so arrays of the structures end up correctly aligned for the system. This allows all systems to assume that the start of structures will always be aligned, causing issues if they aren't. This seems unaffected by the pack or align attributes.

Because of the new case 2, elements of an array in a structure don't necessarily obey the packed or align() attributes specified on the structure, though the start of the array and the field immediately following the array do.

My question is then about how to deal with the structArray in packed_2_s since the size of the array as a whole cannot be guaranteed purely by the packed attribute. Is there some way to guarantee the fixed size of the structArray field as a whole? It should be noted that I can't increase the size of the packed_1_s too much since the data_s struct needs to be kept as small as possible (its replacing Audio/Video data in a streaming scenario).

解决方案

Note the following points about __attribute__((packed)):

• When packed is used in a structure declaration, it will compress its fields such, such that, sizeof(structure) == sizeof(first_member) + ... + sizeof(last_member).

• Here, an array is just one member of the struct. Packing the containing structure of an array will not change the array's size. In fact, the size of (any) array is always sizeof(element) * number_of_elements.

• Similarly, packing a containing structure of an inner structure will not change the size of the inner structure. The size of a structure is completely determined by its declaration, and is the same no matter where you use.

• Packing a structure will make its required alignment one byte (i.e. it can be placed anywhere in memory).

• Packing will introduce alignment issues when accessing the fields of a packed structure. The compiler will account for that when the the fields are accessed directly, but not when they are accessed via pointers. Of course, this does not apply to fields with required alignment one (such as char's or other packed structures). See my answer to a similar question, which includes a program demonstrating the problem with accessing members via pointers.

Finally, to answer the question,

Is there any way to guarantee that the data_s structure will have absolutely no additional space added to it or any of its sub-structures so I don't have compiler dependent shifts in the memory map?

Yes. Declare the structure as packed, and also all structures that it contains, recursively.

Also note that the packed attribute applies to a structure declaration, and not to a type. There's no such thing as packed version of a structure that is declared non-packed. When you use a structure somewhere, it (its members) will be packed if and only if the structure itself was declared packed. This is kind of implied by the fact that the size of a structure is completely determined by its declaration.

UPDATE: For some reason you're still confused about arrays. The solution I provided (declare all structures packed) works with arrays too. For example:

struct elem_struct {
    uint32_t x;
} __attribute__((packed));
// packed guarantees that sizeof(struct elem_struct) = sizeof(uint32_t) = 4

struct array_struct {
    struct elem_struct arr[10];
} __attribute__((packed));
// packed guarantees that sizeof(struct array_struct) =
// = sizeof(struct elem_struct[10]) = 10 * sizeof(struct elem_struct)
// = 10 * 4 = 40

The two additional points you made about arrays are true - but only when the structures are not packed. Packing forces the fields of the struct to be continuous, and this does create alignment issues which, if no packing was used, would be solved by inserting empty space between members and padding the struct (see the point I already raised about alignment).

这篇关于__attribute__((packed)) 对嵌套结构数组的影响?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！