在MPICH中执行混合OpenMP / MPI作业 [英] Executing hybrid OpenMP/MPI jobs in MPICH

问题描述

我很容易就能够启动这些进程，并且他们很容易就能够执行这些进程。做线程，但它们被束缚在与它们的主线程相同的核心上，无论什么类型的 -bind-to 我都试过。

如果我明确地设置 GOMP_CPU_AFFINITY 为 0-15 每个节点有1个进程。

设置 OMP_PROC_BIND = false 没有一个套接字显示效果。

我尝试过很多不同组合的例子

  export OMP_NUM_THREADS = 8 
 export OMP_PROC_BIND =false
 mpiexec.hydra -n 2 -ppn 2 -envall -bind-to numa ./a.out 
  

我得到的所有进程都位于其中一个核心 0-7 上，其中100％核心 8-15 上有多个线程，但其中只有一个接近100％（它们正在等待第一个进程）。

libgomp 缺少相同的尊重子句，因此 英特尔的 KMP_AFFINITY ，你可以通过提供一个包装脚本来从 / proc / PID / status （特定于Linux）：

 ＃！/ bin / sh 
 
 GOMP_CPU_AFFINITY = $（grep ^ Cp us_allowed_list / proc / self / status | grep -Eo'[0-9， - ] +'）
 export GOMP_CPU_AFFINITY 
 exec $ * 
  

这应该适用于 -bind-to numa 然后。

I am struggling to find the proper way to execute a hybrid OpenMP/MPI job with MPICH (hydra).

I am easily able to launch the processes and they do make threads, but they are stuck bound to the same core as their master thread whatever type of -bind-to I tried.

If I explicitly set GOMP_CPU_AFFINITY to 0-15 I get all threads spread but only provided if I have 1 process per node. I don't want that, I want one process per socket.

Setting OMP_PROC_BIND=false does not have a noticeable effect.

An example of many different combinations I tried

export OMP_NUM_THREADS=8
export OMP_PROC_BIND="false"
mpiexec.hydra -n 2 -ppn 2 -envall -bind-to numa  ./a.out

What I get is all process sitting on one of the cores 0-7 with 100% and several threads on cores 8-15 but only one of them close to 100% (they are waiting on the first process).

解决方案

Since libgomp is missing the equivalent of the respect clause of Intel's KMP_AFFINITY, you could hack it around by providing a wrapper script that reads the list of allowed CPUs from /proc/PID/status (Linux-specific):

#!/bin/sh

GOMP_CPU_AFFINITY=$(grep ^Cpus_allowed_list /proc/self/status | grep -Eo '[0-9,-]+')
export GOMP_CPU_AFFINITY
exec $*

This should work with -bind-to numa then.

这篇关于在MPICH中执行混合OpenMP / MPI作业的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！