MPI接收/收集动态矢量长度 [英] MPI Receive/Gather Dynamic Vector Length

问题描述

我有一个存储结构向量的应用程序.这些结构保存有关系统上每个GPU的信息，例如内存和千兆位/秒.每个系统上都有不同数量的GPU.

I have an application that stores a vector of structs. These structs hold information about each GPU on a system like memory and giga-flop/s. There are a different number of GPUs on each system.

我有一个可以同时在多台计算机上运行的程序，我需要收集这些数据.我对MPI还是很陌生，但是大多数时候都可以使用MPI_Gather()，但是我想知道如何收集/接收这些动态大小的向量.

I have a program that runs on multiple machines at once and I need to collect this data. I am very new to MPI but am able to use MPI_Gather() for the most part, however I would like to know how to gather/receive these dynamically sized vectors.

class MachineData
{
    unsigned long hostMemory;
    long cpuCores;
    int cudaDevices;
    public:
    std::vector<NviInfo> nviVec; 
    std::vector<AmdInfo> amdVec;
    ...
};

struct AmdInfo
{
    int platformID;
    int deviceID;
    cl_device_id device;
    long gpuMem;
    float sgflops;
    double dgflops;
};

集群中的每台计算机都填充其MachineData实例.我想收集每个实例，但是我不确定如何收集nviVec和amdVec，因为它们的长度在每台计算机上都不同.

Each machine in a cluster populates its instance of MachineData. I want to gather each of these instances, but I am unsure how to approach gathering nviVec and amdVec since their length varies on each machine.

推荐答案

您可以结合使用MPI_GATHERV和MPI_GATHER来完成此任务. MPI_GATHERV是MPI_GATHER的可变版本，它允许根等级从每个发送过程中收集不同数量的元素.但是，为了让根级别指定这些数字，它必须知道每个级别包含多少个元素.在此之前，可以使用简单的单个元素MPI_GATHER来实现.像这样:

You can use MPI_GATHERV in combination with MPI_GATHER to accomplish that. MPI_GATHERV is the variable version of MPI_GATHER and it allows for the root rank to gather differt number of elements from each sending process. But in order for the root rank to specify these numbers it has to know how many elements each rank is holding. This could be achieved using simple single element MPI_GATHER before that. Something like this:

// To keep things simple: root is fixed to be rank 0 and MPI_COMM_WORLD is used

// Number of MPI processes and current rank
int size, rank;
MPI_Comm_size(MPI_COMM_WORLD, &size);
MPI_Comm_rank(MPI_COMM_WORLD, &rank);

int *counts = new int[size];
int nelements = (int)vector.size();
// Each process tells the root how many elements it holds
MPI_Gather(&nelements, 1, MPI_INT, counts, 1, MPI_INT, 0, MPI_COMM_WORLD);

// Displacements in the receive buffer for MPI_GATHERV
int *disps = new int[size];
// Displacement for the first chunk of data - 0
for (int i = 0; i < size; i++)
   disps[i] = (i > 0) ? (disps[i-1] + counts[i-1]) : 0;

// Place to hold the gathered data
// Allocate at root only
type *alldata = NULL;
if (rank == 0)
  // disps[size-1]+counts[size-1] == total number of elements
  alldata = new int[disps[size-1]+counts[size-1]];
// Collect everything into the root
MPI_Gatherv(vectordata, nelements, datatype,
            alldata, counts, disps, datatype, 0, MPI_COMM_WORLD);

您还应该为结构注册MPI派生数据类型(在上面的代码中为datatype)(二进制发送可以，但是不能移植，并且不能在异构设置中使用).

You should also register MPI derived datatype (datatype in the code above) for the structures (binary sends will work but won't be portable and will not work in heterogeneous setups).

这篇关于MPI接收/收集动态矢量长度的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！