在LAM/MPI集群上安装Rmpi [英] Installing Rmpi on LAM/MPI cluster

问题描述

我正在尝试在LAM MPI群集计算机上安装Rmpi软件包.以前我一直在编译和测试一些东西(mpi4py和小型C ++程序)，所以我确定MPI本身可以工作.但是，链接库时，安装Rmpi软件包失败.

I'm trying to install Rmpi package on a LAM MPI cluster machine. Previously I had been compiling and testing some stuff (mpi4py and small C++ programs) so I'm sure the MPI itself works. However installing Rmpi package fails when linking libraries.

我主要的怀疑是在makefile中调用了gcc而不是mpicc(我试图在配置中查找要更改的行，但到目前为止找不到它).

My main suspect is a call to gcc instead of mpicc in makefile (I'm trying to find the line in configuration to change this but so far could not locate it).

有人在LAM上安装Rmpi有经验吗，您是如何管理的?

Does somebody have experience with installing Rmpi on LAM, and how did you manage that?

建筑

• LAM MPI(或PBS MPI(如果存在)，如何检查?).可以肯定的是，我有一个mpicpp文件，我认为这不是所有mpi实现所共有的
• R-2.15.2
• Rmpi​​_0.6-1
• CentOS 5.5版(最终版)，x86_64

从R调用的命令

install.packages("Rmpi")

登录

trying URL 'http://cran.at.r-project.org/src/contrib/Rmpi_0.6-1.tar.gz'
Content type 'application/x-gzip' length 92977 bytes (90 Kb)
opened URL
==================================================
downloaded 90 Kb

* installing *source* package âRmpiâ ...
** package âRmpiâ successfully unpacked and MD5 sums checked
checking for gcc... gcc -std=gnu99
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc -std=gnu99 accepts -g... yes
checking for gcc -std=gnu99 option to accept ISO C89... none needed
I am here /usr and it is OpenMPI
Trying to find mpi.h ...
Found in /usr/include
Trying to find libmpi.so or libmpich.a ...
Found libmpi in /usr/lib
checking for openpty in -lutil... yes
checking for main in -lpthread... yes
configure: creating ./config.status
config.status: creating src/Makevars
** libs
gcc -std=gnu99 -I/g/software/linux/pack/r-2.15.2/centos-5/lib64/R/include -DNDEBUG -DPACKAGE_NAME=\"\" -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -I/usr/include  -DMPI2 -DOPENMPI -I/usr/local/include    -fpic  -g -O2  -c RegQuery.c -o RegQuery.o
gcc -std=gnu99 -I/g/software/linux/pack/r-2.15.2/centos-5/lib64/R/include -DNDEBUG -DPACKAGE_NAME=\"\" -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -I/usr/include  -DMPI2 -DOPENMPI -I/usr/local/include    -fpic  -g -O2  -c Rmpi.c -o Rmpi.o
gcc -std=gnu99 -I/g/software/linux/pack/r-2.15.2/centos-5/lib64/R/include -DNDEBUG -DPACKAGE_NAME=\"\" -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -I/usr/include  -DMPI2 -DOPENMPI -I/usr/local/include    -fpic  -g -O2  -c conversion.c -o conversion.o
gcc -std=gnu99 -I/g/software/linux/pack/r-2.15.2/centos-5/lib64/R/include -DNDEBUG -DPACKAGE_NAME=\"\" -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -I/usr/include  -DMPI2 -DOPENMPI -I/usr/local/include    -fpic  -g -O2  -c internal.c -o internal.o
gcc -std=gnu99 -shared -L/usr/local/lib64 -o Rmpi.so RegQuery.o Rmpi.o conversion.o internal.o -L/usr/lib -lmpi -lutil -lpthread -L/g/software/linux/pack/r-2.15.2/centos-5/lib64/R/lib -lR
/usr/bin/ld: skipping incompatible /usr/lib/libutil.so when searching for -lutil
/usr/bin/ld: skipping incompatible /usr/lib/libutil.a when searching for -lutil
/usr/bin/ld: skipping incompatible /usr/lib/libpthread.so when searching for -lpthread
/usr/bin/ld: skipping incompatible /usr/lib/libpthread.a when searching for -lpthread
/usr/bin/ld: skipping incompatible /usr/lib/libc.so when searching for -lc
/usr/bin/ld: skipping incompatible /usr/lib/libc.a when searching for -lc
installing to /home/andrejev/R/x86_64-unknown-linux-gnu-library/2.15/Rmpi/libs
** R
** demo
** inst
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded
Error : .onLoad failed in loadNamespace() for 'Rmpi', details:
  call: dyn.load(file, DLLpath = DLLpath, ...)
  error: unable to load shared object '/home/andrejev/R/x86_64-unknown-linux-gnu-library/2.15/Rmpi/libs/Rmpi.so':
  /usr/lib/libmpi.so.0: undefined symbol: _kio
Error: loading failed
Execution halted
ERROR: loading failed
* removing â/home/andrejev/R/x86_64-unknown-linux-gnu-library/2.15/Rmpiâ

The downloaded source packages are in
        â/tmp/Rtmp5qzeqp/downloaded_packagesâ
Warning message:
In install.packages("Rmpi") :
  installation of package âRmpiâ had non-zero exit status

推荐答案

我们有一个非常相似的设置.我最近还没有触及到它，无法与您的具体情况谈一谈，但是我可以传递一些使我们起步运行的命令...

We have a very similar setup. I haven't touched it recently enough to be able to speak to your particular situation, but I can pass along a few of the commands that got us up-and-running...

正如Dirk所提到的，对我们来说，第一步是使OpenMPI运行.设置完成后，您需要确保正确设置LD_LIBRARY_PATH:

The first step for us, as Dirk mentioned, was to get OpenMPI running. Once that's setup, you'll need to be sure your LD_LIBRARY_PATH is properly set:

export LD_LIBRARY_PATH=/opt/openmpiv2/lib/:$LD_LIBRARY_PATH

最终为我们安装Rmpi的命令(运行上述命令并启动R之后)是:

The command that finally got Rmpi installed for us (after running the above command and starting R) was:

install.packages("Rmpi", configure.args = c("--with-Rmpi-include=/opt/openmpiv2/include/", "--with-Rmpi-libpath=/opt/openmpiv2/lib/", "--with-Rmpi-type=OPENMPI", "--with-mpi=/opt/openmpiv2/"))

哪个为我们成功安装了软件包.然后，我们可以使用以下命令来运行作业:

Which installed the package successfully for us. Then we can run jobs using commands such as:

qsub -I -l nodes=2:ppn=12
/opt/openmpiv2/bin/mpirun --hostfile $PBS_NODEFILE -n 1 R --slave -f rmpitest.R

这些命令可能对我们的设置非常特定，以至于它们可能没有用，但是希望此处的某些命令可以为您节省一些时间！

These commands may be so specific to our setup that they may not be useful, but hopefully something in here can save you some time!

如@Sergej在下面提出的，一个更通用的解决方案可能是:

As @Sergej proposes below, a more general solution may be:

install.packages(
  "Rmpi", 
  configure.args = c(
   "--with-Rmpi-include=/usr/include/", # This is where LAM's mpi.h is located
   "--with-Rmpi-libpath=/usr/lib/",     # This is where liblam.so is located (actually as I type it mine was located in /usr/lib64/liblam.so.0, so maybe this is not needed at all)
   "--with-Rmpi-type=LAM"               # This says that the type is LAM (there is also OPENMPI and MPICH)
))

我们的设置:RHEL 5，Rmpi 0.5-9，OpenMPI 1.4.3.，R 2.15.0

Our setup: RHEL 5, Rmpi 0.5-9, OpenMPI 1.4.3., R 2.15.0

这篇关于在LAM/MPI集群上安装Rmpi的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！