如何从Python调用MPI .so文件？ [英] How can I call an MPI .so file from Python?

问题描述

我有一个共享的目标文件，其中包含已编译的C ++ MPI Hello World代码。当我尝试使用ctypes从Python调用它时，我得到了一些无用的错误列表。

I have a shared object file that contains compiled C++ MPI Hello World code. When I try to call it from Python using ctypes, I get a list of rather unhelpful errors.

mpiHello.cpp：

mpiHello.cpp:

#include <mpi.h>

extern "C"
void mpiHello() {

    int rank, size;

    MPI_Init(NULL, NULL);

    MPI_Comm_rank(MPI_COMM_WORLD, &rank);
    MPI_Comm_size(MPI_COMM_WORLD, &size);

    std::cout << "Hello world! I am " << rank << " of " << size << std::endl;

    MPI_Finalize();

}

编译命令：
mpic ++ -fPIC -o mpi.so mpiHello.cpp -shared

Python调用：

    def __init__(self):
        self.dll = None
        _DIRNAME = os.path.dirname(__file__)
        try:  # Windows
            self.dll = ctypes.CDLL(os.path.join(_DIRNAME, "mpi.dll"))
        except OSError:  # Linux
            self.dll = ctypes.cdll.LoadLibrary(os.path.join(_DIRNAME, "mpi.so"))
        finally:
            self.dll.mpiHello.argtypes = []

    def execute(self):
        self.dll.mpiHello()

_mpi = mpi()
_mpi.execute()

[<user>-OptiPlex-7050:09468] mca_base_component_repository_open: unable to open mca_shmem_mmap: /usr/lib/x86_64-linux-gnu/openmpi/lib/openmpi/mca_shmem_mmap.so: undefined symbol: opal_show_help (ignored)
[<user>-OptiPlex-7050:09468] mca_base_component_repository_open: unable to open mca_shmem_sysv: /usr/lib/x86_64-linux-gnu/openmpi/lib/openmpi/mca_shmem_sysv.so: undefined symbol: opal_show_help (ignored)
[<user>-OptiPlex-7050:09468] mca_base_component_repository_open: unable to open mca_shmem_posix: /usr/lib/x86_64-linux-gnu/openmpi/lib/openmpi/mca_shmem_posix.so: undefined symbol: opal_shmem_base_framework (ignored)
--------------------------------------------------------------------------
It looks like opal_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during opal_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  opal_shmem_base_select failed
  --> Returned value -1 instead of OPAL_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like orte_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during orte_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  opal_init failed
  --> Returned value Error (-1) instead of ORTE_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like MPI_INIT failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during MPI_INIT; some of which are due to configuration or environment
problems.  This failure appears to be an internal failure; here's some
additional information (which may only be relevant to an Open MPI
developer):

  ompi_mpi_init: ompi_rte_init failed
  --> Returned "Error" (-1) instead of "Success" (0)
--------------------------------------------------------------------------
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[<user>-OptiPlex-7050:9468] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!

Process finished with exit code 1

我希望代码显示4行带有 Hello world！我是...但是我只是得到了错误。我们将不胜感激！

I would expect the code to display 4 lines with "Hello world! I am... but I just get the errors. Any help would be greatly appreciated!

推荐答案

在Linux上几乎没有任何事可做，实际上您的错误缺少括号。

On Linux there pretty much nothing to do, in fact your mistake are missing brackets.

#include "mpiHello.h" 

extern "C" {
    void mpiHello() {
        // Code
    }
}

与 mpiHello.h 文件相同：

#pragma once
#include <mpi.h>

extern "C" {
    void mpiHello();
}

要通过python使用共享库，我通常使用宏来处理导出范围，它将与Windows（ .dll ）和Linux（ .so ）一起使用。

To work with shared libraries with python. I generally use a macro to handle the export scope, it will work with Windows (.dll) and Linux (.so).

mpi.cpp ：

#include "mpiHello.h"

extern "C" {
    MYEXPORT void mpiHello() {
        // Code
    }
}

mpiHello.h ：

#pragma once
#include <mpi.h>
#if _WIN32
#define MYEXPORT __declspec(dllexport)
#else
#define MYEXPORT 
#endif

extern "C" {
    MYEXPORT void mpiHello();
}

如果要使用返回值或参数创建新函数，请小心。您需要在python文件中指定它们。

Be carefull if you want to create a new fonction with a return value or arguments. You will need to specify them in your python file.

如果您有新的功能，则 int myFonction（char * str1，char * str2，int pos），那么您将在python中有以下声明：

If you have a new fonction int myFonction(char* str1, char* str2, int pos), then you will have the following déclaration in python:

dll.myFonction.argtypes = [ctype.c_char_p, ctype.c_char_p, ctype.c_int]
dll.myFunction.restype = ctype.c_int

此外，您将必须使用C类型参数来提供新功能，因此必须将python值转换为C值。
以下是将python列表和python int转换为ctypes的示例：

Furthermore, you will have to feed your new fonctions with C type arguments, thus you will have to convert your python values in C values. Here is an example to convert a python list and a python int to ctypes:

pyhton_string = "Hello Word"
python_int = 42

c_string = (ctype.c_char* len(pyhton_string))(*pyhton_string)
c_int = ctype.c_int(python_int)

这篇关于如何从Python调用MPI .so文件？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！