如何使用MS HPC Server 2008 R2的MPI堆栈成功编译mpi4py? [英] How to successfully compile mpi4py using MS HPC Server 2008 R2's MPI stack?

问题描述

故事就这样了:我需要一个用于Python的MPI包装器.

So the story goes: I need a MPI wrapper for Python.

我知道有 mpi4py .对于当前的工作，我(主要)使用Python和Windows，我想使用Microsoft HPC群集包，可以访问一些运行Win 2008 Server的强大"计算机.只需提一下，除了Win经验，我在MPI和其他方面确实有* nix的经验，但这是解决该问题的有意思的地方.

I know there's mpi4py. For the current work I (mostly) use Python and Windows, I'd like to use the Microsoft HPC Cluster Pack, having access to a few pretty "strong" machines running Win 2008 Server. Just to mention, besides Win-experience, I do have a bit of *nix experience with MPI and stuff, but that's pretty moot point for this problem.

当我遇到适用于Visual Studio的Python工具时，我重新对mpi4py感兴趣.那是一些非常棒的东西.任何喜欢Visual Studio和Python的人都应该尝试一下.做得好，调试器很棒.

My interest in mpi4py was renewed when I ran into Python Tools for Visual Studio. That's some seriously great stuff. Anyone that's a fan of Visual Studio and Python should try it. Good work, and a great debugger.

PTVS的文档页面指出mpi4py的安装非常简单...对于ActiveState Python来说，这似乎是事实.但是，如果您不使用ActiveState的Python，而是使用python.org中的正常" Python发行版，那么您似乎有点不走运.

The doc pages of PTVS state the installation of mpi4py is easy... and for ActiveState Python it seems to be true. However, if you don't use ActiveState's Python and instead use the "normal" Python distribution from python.org, you seem to be a bit out of luck.

我的开发机器是带64位和32位版本的带Win7 64位和Python 2.6的笔记本电脑. 我已经安装了MS HPC Pack 2008 R2 MS MPI和SDK.我安装了Visual Studio 2008和2010，对所有内容进行了认真的修补.

My development machine is a laptop w/ Win7 64bit and Python 2.6, both in 64-bit and 32-bit flavors. I have installed the MS HPC Pack 2008 R2 MS MPI and SDK. I have Visual Studio 2008 and 2010, everything dutifully patched.

没有二进制安装程序，并且知道Unix MPI如何对其链接到的MPI版本非常挑剔，所以我想构建自己的mpi4py. mpi4py基本上依赖于具有将python调用绑定到MPI库的MPI .dll(实际上是.pyd).

There's no binary installer and knowing how Unix MPIs can be exceedingly picky wrt to the MPI version they're linked with, I wanted to build my own mpi4py. mpi4py basically relies on having a MPI .dll (.pyd actually) that binds the python calls to MPI libs.

mpi4py的easy_install安装和该库的构建失败-无法指向MPI库.好的，没问题，我下载了mpi4py tarball，将其解压缩并更改了 mpi.cfg 文件，使其指向正确的文件夹:

easy_installing of mpi4py and building of that library has failed - not being able to point to the MPI libs. OK, no problem, I downloaded the mpi4py tarball, extracted it and altered the mpi.cfg file so that it points to the correct folders:

# Microsoft MPI example
# ---------------------
[msmpi]
define_macros = MS_MPI=1
mpi_dir = $CCP_HOME
include_dirs = %(mpi_dir)s\Inc
libraries = msmpi
library_dirs = %(mpi_dir)s\lib\i386

MS MPI安装程序注册一个环境变量CCP_HOME，该环境变量指向Pack的确切安装位置.名称"CCP"必须从被称为Microsoft Compute Cluster Pack的日期开始保留下来.已将其传递给原始的mpi4py开发人员.

The MS MPI installer registers an environment variable CCP_HOME pointing to the exact install location of the Pack. The name "CCP" must be leftover from the days it was called Microsoft Compute Cluster Pack). Got to pass this to the original mpi4py developer.

此后，编译通过，但我无法链接-共有三个未解决的外部变量:

After this, the compilation passes fine, but I can't link - there are three unresolved externals:

MPI.obj : error LNK2019: unresolved external symbol _MPI_Type_create_f90_integer@8 referenced in ...
MPI.obj : error LNK2019: unresolved external symbol _MPI_Type_create_f90_real@12 ...
MPI.obj : error LNK2019: unresolved external symbol _MPI_Type_create_f90_complex@12 ...

似乎HPC 2008 R2中的MS MPI msmpi.lib没有实现这些，因此我无法构建MPI.pyd.

Seems that the MS MPI msmpi.lib from HPC 2008 R2 does not implement these, so I can't build the MPI.pyd.

我可以尝试在mpi4py C源文件中将它们注释掉，但是我认为这不是正确的路径.

I could try to comment out these in the mpi4py C source file, but I don't think this is the right path.

提前谢谢！

推荐答案

我正在与@Hrvoje和源代码的当前维护者进行交谈. https://code.google.com/p/mpi4py/

I was in the talk with @Hrvoje and the current maintainer of the source code at https://code.google.com/p/mpi4py/

感谢大家的帮助.
我使用了Visual Studio 2012，Python 2.7.3(64位)和MPI4PY 1.3

Thanks guys for all your help.
I used Visual Studio 2012, Python 2.7.3 (64bit) and MPI4PY 1.3

以下是更改:

• mpi标头和库现在位于其他位置(MS HPC 2008 R2)，因此我的mpi.cfg的msmpi部分现在看起来像这样:

• mpi headers and libs are now in a different location (MS HPC 2008 R2) so my msmpi part of the mpi.cfg looks like this now:

[msmpi]
mpi_dir = $ProgramFiles\Microsoft HPC Pack 2008 R2
include_dirs = %(mpi_dir)s\inc
libraries = msmpi
#library_dirs = %(mpi_dir)s\lib\i386
library_dirs = %(mpi_dir)s\lib\amd64

因为Python构建环境正在寻找Visual Studio 2008，所以我不得不手动添加它正在寻找的变量. 它实际上指向VS 2012目录，但是所有构建工具都兼容，因此可以正常工作. 这就是我添加的内容.

Because the Python build environment looks for Visual Studio 2008 I had to manually add the variable it was looking for. It actually points to a VS 2012 directory, but all the build tools are compatible so it worked. This is what I've added.

VS90COMNTOOLS = C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\Tools\

除此之外，没有任何更改，setup.py生成.exe和.msi都没有任何问题.

Other than that, there were no changes and the setup.py produced both .exe and .msi without any issues.

这篇关于如何使用MS HPC Server 2008 R2的MPI堆栈成功编译mpi4py?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！