如何在一次 PBS 作业提交中运行多个命令 [英] How to run several commands in one PBS job submission

问题描述

我写了一个只需要 1-4 个 CPU 的代码.但是当我在集群上提交作业时，我必须至少采用一个节点，每个作业具有 16 个内核.所以我想对我提交的每个作业在每个节点上运行几次模拟.我想知道是否有办法在一项工作中并行提交模拟.

I have written a code that takes only 1-4 cpus. But when I submit a job on the cluster, I have to take at least one node with 16 cores per job. So I want to run several simulations on each node with each job I submit. I was wondering if there is a way to submit the simulations in parallel in one job.

这是一个例子:我的代码需要 4 个 CPU.我为一个节点提交了一个作业，我希望该节点运行我的代码的 4 个实例(每个实例具有不同的参数)以获取所有 16 个内核.

Here's an example: My code takes 4 cpus. I submit a job for one node, and I want the node to run 4 instances of my code (each instance has different parameters) to take all the 16 cores.

推荐答案

是的，当然；通常，此类系统会提供有关如何执行此操作的说明，像这些.

Yes, of course; generally such systems will have instructions for how to do this, like these.

如果您有(例如)4 个 4-cpu 作业，并且您知道每个作业将花费相同的时间，并且(例如)您希望它们在 4 个不同的目录中运行(因此输出文件更容易跟踪), 使用 shell &符号在后台运行它们，然后等待所有后台任务完成:

If you have (say) 4x 4-cpu jobs that you know will each take the same amount of time, and (say) you want them to run in 4 different directories (so the output files are easier to keep track of), use the shell ampersand to run them each in the background and then wait for all background tasks to finish:

(cd jobdir1; myexecutable argument1 argument2) &
(cd jobdir2; myexecutable argument1 argument2) &
(cd jobdir3; myexecutable argument1 argument2) &
(cd jobdir4; myexecutable argument1 argument2) &
wait

(其中 myexecutable argument1 argument2 只是您通常运行程序的占位符；如果您使用 mpiexec 或类似的东西，那就像您一样进入那里通常会使用它.如果您使用的是 OpenMP，则可以在上面的第一行之前导出环境变量 OMP_NUM_THREADS.

(where myexecutable argument1 argument2 is just a place holder for however you usually run your program; if you use mpiexec or something similar, that goes in there just as you'd normally use it. If you're using OpenMP, you can export the environment variable OMP_NUM_THREADS before the first line above.

如果您有许多任务都不会花费相同的时间，那么最简单的方法是分配比上面(例如)4 个工作更多的任务，并使用像 gnu parallel 根据需要启动作业，如 这个答案.

If you have a number of tasks that won't all take the same length of time, it's easiest to assign well more than the (say) 4 jobs above and let a tool like gnu parallel launch the jobs as necessary, as described in this answer.

这篇关于如何在一次 PBS 作业提交中运行多个命令的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！