重定向通过MPI_Comm_spawn生成的子级的stdout [英] Redirecting stdout from children spawned via MPI_Comm_spawn

问题描述

我有一个主应用程序，该应用程序生成一个工作程序，而该工作程序本身又生成了两个从属程序.从属应用程序将其输出写入stdout.我的想法是将stdout绑定到工作程序应用程序中的其他流，以便能够将从属设备的输出存储在变量中，并将其发送到处理输出的主服务器.但是，从站的stdout无法正确重定向，并仍显示在控制台上.工作器应用程序中的缓冲区保持为空.我是否缺少某些东西，或者这是我做不到的方式?如果这样的话，任何有关如何以不同方式处理此问题的建议都将受到赞赏.我在Gentoo上使用Open MPI 1.6.5，这是我的应用程序的源代码:

I have a master application which spawns a worker which itself spawns two slaves. The slave application writes its output to stdout. My idea was to bind stdout to a different stream in the worker application for being able to store the output of the slaves in a variable and send it over to the master, which handles the output. However, stdout of the slaves does not get redirected properly and still appears on the console. The buffer in the worker application stays empty. Am I missing something or is this not possible in the way I do it? If so, any recommendations on how to handle this issue in a different manner a greatly appreciated. I'm using Open MPI 1.6.5 on Gentoo and here's the source code of my applications:

master.cpp

master.cpp

#include <mpi.h>
#include <iostream>

using namespace std;

int main(int argc, char *argv[])
{
    char appExe[] = "worker";
    char *appArg[] = {NULL};
    int maxProcs = 1;
    int myRank; 
    MPI_Comm childComm;
    int spawnError;

    // Initialize
    MPI_Init(&argc, &argv);

    // Rank 
    MPI_Comm_rank(MPI_COMM_WORLD, &myRank);

    // Spawn application    
    MPI_Comm_spawn(appExe, appArg, maxProcs, MPI_INFO_NULL, myRank, MPI_COMM_SELF, &childComm, &spawnError);

    // Receive length of message from worker
    int len;
    MPI_Recv(&len, 1, MPI_INT, 0, MPI_ANY_TAG, childComm, MPI_STATUS_IGNORE);
    // Receive actual message from worker
    char *buf = new char[len];
    MPI_Recv(buf, len, MPI_CHAR, 0, MPI_ANY_TAG, childComm, MPI_STATUS_IGNORE);
    cout << "master: Got the following from worker: " << buf << endl;

    // Finalize
    MPI_Finalize();

    return 0;
}

worker.cpp

worker.cpp

#include "mpi.h"
#include <iostream>
#include <string>
#include <sstream>

using namespace std;

int main(int argc, char *argv[])
{
    char appExe[] = "slave";
    char *appArg[] = {NULL};
    int maxProcs = 2;
    int myRank, parentRank; 
    MPI_Comm childComm, parentComm;
    int spawnError[maxProcs];

    // Initialize
    MPI_Init(&argc, &argv);

    // Rank
    MPI_Comm_rank(MPI_COMM_WORLD, &myRank);

    // Get parent
    MPI_Comm_get_parent(&parentComm);

    // Bind stdout to new_buffer
    stringstream new_buffer;
    streambuf *old_buffer = cout.rdbuf(new_buffer.rdbuf());  

    // Spawn application    
    MPI_Comm_spawn(appExe, appArg, maxProcs, MPI_INFO_NULL, myRank, MPI_COMM_SELF, &childComm, spawnError);

    // Enter barrier
    MPI_Barrier(childComm);

    // Reset stdout to old_buffer
    cout.rdbuf(old_buffer);

    // Make a string
    string tmp = new_buffer.str();
    // Make a character array from string
    const char* cstr = tmp.c_str();
    cout << "worker: Got the following from slaves: " << cstr << endl;

    // Send length of message to master   
    int len = sizeof(cstr);
    MPI_Send(&len, 1, MPI_INT, 0, 0, parentComm);
    // Send actual message
    MPI_Send(&cstr, len, MPI_CHAR, 0, 0, parentComm);

    // Finalize
    MPI_Finalize();

    return 0;
}

slave.cpp

slave.cpp

#include <mpi.h>
#include <iostream>

using namespace std;

int main(int argc, char *argv[])
{
    MPI_Comm parent;

    // Initialize
    MPI_Init(&argc, &argv);

    // Get parent
    MPI_Comm_get_parent(&parent);

    // Say hello
    cout << "slave: Hi there!" << endl;

    // Enter barrier
    if (parent != MPI_COMM_NULL)
        MPI_Barrier(parent);

    // Finalize
    MPI_Finalize();

    return 0;
}

推荐答案

MPI中的作业生成是在同一"Universe"中发生的，通常是由用于启动初始MPI作业的同一应用程序启动器执行的.在Open MPI中，这将是orterun(mpiexec和mpirun都是到orterun的符号链接). I/O重定向由ORTE(开放MPI运行时环境，MPI库的一部分)执行，并将每个MPI进程的标准输出发送到orterun，然后将所有内容混合并将其显示到其控制台输出或在有输出重定向的情况下将其保存到文件中.除非产生的作业专门将其输出写入文件，否则父级将无法拦截该输出.

Job spawning in MPI happens in the same "universe" and is usually performed by the same application launcher that is being used to launch the initial MPI job. In Open MPI that would be orterun (mpiexec and mpirun are both symlinks to orterun). I/O redirection is performed by the ORTE (the Open MPI run-time environment, part of the MPI library) and it sends the standard output of each MPI process to orterun, which then mixes everything and displays it to its console output or saves it to a file if output redirection is in place. Unless a spawned job specifically writes its output to a file, the parent has no way to intercept that output.

在父作业和衍生的作业之间进行通信的另一种(也是唯一的)MPI兼容方式是使用MPI消息传递.您可以实现自己的C ++输入和输出流类，这些类使用MPI消息通过互连器传输数据.

The other (and only) MPI-compliant way to communicate between the parent job and the spawned jobs is to use MPI message passing. You can implement your own C++ input and output stream classes that use MPI messages to transmit data over the intercommunicator.

这篇关于重定向通过MPI_Comm_spawn生成的子级的stdout的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！