使用boost :: mpi在mpi中的消息大小是否有限制? [英] Is there a limit for the message size in mpi using boost::mpi?

问题描述

我目前正在openMPI之上使用boost :: mpi编写模拟，一切工作都很好.但是，一旦我扩展了系统，因此不得不发送更大的std :: vectors，我就会出错.

I'm currently writing a simulation using boost::mpi on top of openMPI and everything works great. However once I scale up the system and therefore have to send larger std::vectors I get errors.

我已将问题简化为以下问题:

I've reduced the issue to the following problem:

#include <boost/mpi.hpp>
#include <boost/mpi/environment.hpp>
#include <boost/mpi/communicator.hpp>
#include <boost/serialization/vector.hpp>
#include <iostream>
#include <vector>
namespace mpi = boost::mpi;

int main() {
    mpi::environment env;
    mpi::communicator world;

    std::vector<char> a;
    std::vector<char> b;
    if (world.rank() == 0) {
        for (size_t i = 1; i < 1E10; i *= 2) {
            a.resize(i);
            std::cout << "a " << a.size();
            world.isend(0, 0, a);
            world.recv(0, 0, b);
            std::cout << "\tB " << b.size() << std::endl;
        }
    }
    return 0;
}

打印出:

a 1 B 1
a 2 B 2
a 4 B 4
....
a 16384 B 16384
a 32768 B 32768
a 65536 B 65536
a 131072    B 0
a 262144    B 0
a 524288    B 0
a 1048576   B 0
a 2097152   B 0

我知道一个mpi消息的大小是有限制的，但是65kB对我来说似乎有点低. 有发送更大消息的方法吗?

I'm aware that there is a limit to a mpi message size, but 65kB seems a little low to me. Is there a way of sending larger messages?

推荐答案

消息大小的限制与MPI_Send的限制相同:INT_MAX.

The limit of the message size is the same as for MPI_Send: INT_MAX.

问题是在下一次迭代中调整向量a的大小之前，您没有等待isend完成.这意味着isend将由于向量a中的重新分配而读取无效数据.请注意，缓冲区a是通过引用传递给boost::mpi的，因此您不允许在isend操作完成之前更改缓冲区a.

The issue is that you are not waiting for the isend to finish before resizing the vector a in the next iteration. This means that isend will read invalid data due to the reallocations in the vector a. Note that buffer a is passed by reference to boost::mpi and you are thus not allowed to change the buffer a until the isend operation has finished.

如果使用valgrind运行程序，则只要i = 131072，您就会看到无效的读取.

If you run your program with valgrind, you will see invalid reads as soon as i = 131072.

您的程序可以工作到65536字节的原因是，如果OpenMPI小于组件btl_eager_limit，它们将直接发送消息.对于self组件(发送到自己的进程)，恰好是128*1024字节.由于boost::serialization将std::vector的大小添加到字节流中，因此一旦使用128*1024 = 131072作为输入大小，就会超过该eager_limit.

The reason your program works till 65536 bytes, is that OpenMPI will send messages directly if they are smaller than the components btl_eager_limit. For the self component (sending to the own process), this happens to be 128*1024 bytes. Since boost::serialization adds the size of the std::vector to the byte stream, you exceed this eager_limit as soon as you use 128*1024 = 131072 as your input size.

要修复代码，请保存isend()中的boost::mpi::request返回值，然后在循环末尾添加wait():

To fix your code, save the boost::mpi::request return value from isend() and then add wait() to the end of the loop:

#include <boost/mpi.hpp>
#include <boost/mpi/environment.hpp>
#include <boost/mpi/communicator.hpp>
#include <boost/serialization/vector.hpp>
#include <iostream>
#include <vector>
namespace mpi = boost::mpi;

int main() {
    mpi::environment env;
    mpi::communicator world;

    std::vector<char> a;
    std::vector<char> b;
    if (world.rank() == 0) {
        for (size_t i = 1; i < 1E9; i *= 2) {
            a.resize(i);
            std::cout << "a " << a.size();
            mpi::request req = world.isend(0, 0, a);
            world.recv(0, 0, b);
            std::cout << "\tB " << b.size() << std::endl;
            req.wait();
        }
    }
    return 0;
}

这篇关于使用boost :: mpi在mpi中的消息大小是否有限制?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！