具有动态分配成员的动态分配结构的 MPI 派生数据类型 [英] MPI derived datatype for dynamically allocated structs with dynamically allocated member

问题描述

有一个动态分配的结构体:

There is a dynamically allocated struct:

TYPE Struct    
    INTEGER :: N
    REAL*8 :: A
    REAL*8,ALLOCATABLE :: B(:)
END TYPE Struct

它有一个动态分配的成员:B(:)

and it has a dynamically allocated memeber : B(:)

当我尝试使用 MPI_TYPE_CREATE_STRUCT 为此类 Struct 创建派生数据类型时，不同的 CPU 会创建不一致的派生数据类型.这是因为相对于第一个成员 Struct%N，Struct%B(:) 可能位于不同的内存位置，在不同的 CPU 上.

When I try to use MPI_TYPE_CREATE_STRUCT to create a derived datatype for such Struct, it happens that differen CPUs create inconsistent derived datatypes. This is because Struct%B(:) might be located in different memory locations relative to the first memeber Struct%N, on different CPUs.

那么 MPI_SEND(Struct,...) 不会成功...

Then MPI_SEND( Struct,...) won't succeed...

那么，如果我真的想使用 MPI 派生数据类型发送这个 Struct，如何解决这样的问题?

So, how could such problem be solved if I really want to send this Struct using MPI derived datatype?

还是禁止这种派生数据类型?

Or such kind of derived datatype is forbidden?

推荐答案

要发送该结构的 一个 元素，只需 照常进行，使用MPI_TYPE_CREATE_STRUCT创建结构化数据类型.根据程序的堆和堆栈的相对位置，B(1) 相对于 N 的偏移量最终可能是一个巨大的正数或负数，但这在大多数 Unix 平台上没有问题，并且该数字应在 INTEGER(KIND=MPI_ADDRESS_KIND) 的范围内.

To send one element of that struct, simply proceed as usual by creating a structured datatype using MPI_TYPE_CREATE_STRUCT. Depending on how the heap and the stack of the program are located in respect to each other, the offset of B(1) relative to N could end up being a a huge positive or negative number, but that is of no issue on most Unix platforms and the number should fit in the range of INTEGER(KIND=MPI_ADDRESS_KIND).

重要提示:结构的单独实例很可能相对于 N 具有不同的 B 偏移量，因此 MPI 数据类型只能是用于发送特定记录，用于在数据类型的构造过程中获取偏移量.

Important: Separate instances of the structure will most likely have different offsets of B relative to N, therefore the MPI datatype can only be used for sending the specific record that was used for obtaining the offsets during the construction of the datatype.

当我尝试使用 MPI_TYPE_CREATE_STRUCT 为此类 Struct 创建派生数据类型时，不同的 CPU 会创建不一致的派生数据类型.这是因为相对于第一个成员 Struct%N，Struct%B(:) 可能位于不同的内存位置，在不同的 CPU 上.

When I try to use MPI_TYPE_CREATE_STRUCT to create a derived datatype for such Struct, it happens that differen CPUs create inconsistent derived datatypes. This is because Struct%B(:) might be located in different memory locations relative to the first memeber Struct%N, on different CPUs.

这不是问题.通信操作双方的 MPI 数据类型只能是全等的，这意味着它们应该由相同顺序的相同基本数据类型组成.每个元素的偏移量无关. 换句话说，只要发送者和接收者在 MPI_TYPE_CREATE_STRUCT 调用中指定相同类型和数量的数据元素，程序将正常运行.

This is a non-issue. The MPI datatypes on both sides of the communication operation must only be congruent, which means that they should consist of the same basic datatypes in the same sequence. The offset of each element is irrelevant. In other words, as long as both the sender and the receiver specify the same types and number of data elements in the MPI_TYPE_CREATE_STRUCT call, the program will function correctly.

要发送多个元素，事情会变得有点复杂.有两种解决方案:

To send more than one element, things get a bit complicated. There are two solutions:

使用 MPI_PACK 序列化发送方的数据，使用 MPI_UNPACK 反序列化接收方的数据.由于打包和解包需要额外的缓冲区空间，这会使程序的内存需求增加一倍.

Use MPI_PACK to serialise the data on the sender side and MPI_UNPACK to deserialise it on the receiver side. As packing and unpacking require additional buffer space, this doubles the memory requirements of the program.

或

为每条记录创建一个单独的 MPI 结构数据类型，然后创建一个组合所有记录的结构数据类型.下面是一个示例，说明如何发送包含两个这样的结构的数组，一个在 B 中有 10 个元素，一个在 B 中有 20 个元素:

Create a separate MPI structure datatype for each record and then create a structure datatype that combines all records. Here is an example of how to send an array of two such structures, one with 10 elements in B and one with 20:

TYPE(Struct) :: Structs(2)

ALLOCATE(Structs(1)%B(10))
ALLOCATE(Structs(2)%B(20))

! (1) Create a separate structure datatype for each record
DO i=1,2
  CALL MPI_GET_ADDRESS(Structs(i)%N,    POS_(1), IError)
  CALL MPI_GET_ADDRESS(Structs(i)%A,    POS_(2), IError)
  CALL MPI_GET_ADDRESS(Structs(i)%B(1), POS_(3), IError)
  Offsets = POS_ - POS_(1)

  Types(1) = MPI_INTEGER
  Types(2) = MPI_REAL8
  Types(3) = MPI_REAL8

  Blocks(1) = 1
  Blocks(2) = 1
  Blocks(3) = i * 10

  CALL MPI_TYPE_CREATE_STRUCT(3, Blocks, Offsets, Types, Elem_Type(i), IError)
END DO

! (2) Create a structure of structures that describes the whole array
CALL MPI_GET_ADDRESS(Structs(1)%N, POS_(1), IError)
CALL MPI_GET_ADDRESS(Structs(2)%N, POS_(2), IError)
Offsets = POS_ - POS_(1)

Types(1) = Elem_Type(1)
Types(2) = Elem_Type(2)

Blocks(1) = 1
Blocks(2) = 1

CALL MPI_TYPE_CREATE_STRUCT(2, Blocks, Offsets, Types, TwoElem_Type, IError)
CALL MPI_TYPE_COMMIT(TwoElem_Type, IError)

! (2.1) Free the intermediate datatypes
DO i=1,2
  CALL MPI_TYPE_FREE(Elem_Type(i), IError)
END DO

! (3) Send the array
CALL MPI_SEND(Structs(1)%N, 1, TwoElem_Type, ...)

请注意，尽管构造 MPI 数据类型是一项相对便宜的操作，但您不应使用上述过程发送例如结构化类型的 1000000 个实例.此外，MPI 数据类型描述符存在于库管理的内存中，及时释放不再需要的数据类型很重要.

Note that though constructing MPI datatypes is a relatively cheap operation, you should not use the procedure described above to send e.g. 1000000 instances of the structured type. Also, MPI datatype descriptors live in memory managed by the library and the timely deallocation of datatypes that are no longer needed is important.

这篇关于具有动态分配成员的动态分配结构的 MPI 派生数据类型的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！