在MPI流程之间传递可变长度的结构 [英] Passing variable-length structures between MPI processes

问题描述

我需要MPI_Gatherv()一些int/string对.假设每对看起来像这样:

I need to MPI_Gatherv() a number of int/string pairs. Let's say each pair looks like this:

struct Pair {
  int x;
  unsigned s_len;
  char s[1]; // variable-length string of s_len chars
};

如何为配对定义合适的MPI数据类型?

How to define an appropriate MPI datatype for Pair?

推荐答案

简而言之，理论上不可能发送一个大小可变的消息并将其接收到大小合适的缓冲区中.您要么必须发送带有每个字符串大小的第一条消息，然后发送带有字符串本身的第二条消息，要么将该元信息编码为有效负载并使用静态接收缓冲区.

In short, it's theoretically impossible to send one message of variable size and receive it into a buffer of the perfect size. You'll either have to send a first message with the sizes of each string and then a second message with the strings themselves, or encode that metainfo into the payload and use a static receiving buffer.

如果您只必须发送一条消息，那么我将放弃为Pair定义一种数据类型:相反，我将为整个有效负载创建一种数据类型，并将所有数据转储到一个连续的，无类型的包中.然后，在接收端，您可以对其进行迭代，为每个字符串分配所需的确切空间，并将其填满.让我整理一下ASCII图进行说明.这将是您的有效载荷:

If you must send only one message, then I'd forgo defining a datatype for Pair: instead, I'd create a datatype for the entire payload and dump all the data into one contiguous, untyped package. Then at the receiving end you could iterate over it, allocating the exact amount of space necessary for each string and filling it up. Let me whip up an ASCII diagram to illustrate. This would be your payload:

| ..x1 .. | ..s_len1 .. | .... string1 .... | ..x2 .. | ..s_len2 .. | .string2 .... x3 .. | .. s_len3 .. | .... string3 ....... | ...

|..x1..|..s_len1..|....string1....|..x2..|..s_len2..|.string2.|..x3..|..s_len3..|.......string3.......|...

您将整个东西作为一个单元发送(例如，一个MPI_BYTE数组)，然后接收器将像这样将其拆包:

You send the whole thing as one unit (e.g. an array of MPI_BYTE), then the receiver would unpack it something like this:

while (buffer is not empty)
{
    read x;
    read s_len;
    allocate s_len characters;
    move s_len characters from buffer to allocated space;
}

但是请注意，仅当整数和char的数据表示在发送和接收系统上相同时，此解决方案才有效.

Note however that this solution only works if the data representation of integers and chars is the same on the sending and receiving systems.

这篇关于在MPI流程之间传递可变长度的结构的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！