相同脚本但具有不同输入字符串参数的SLURM抽签作业数组并行运行 [英] SLURM sbatch job array for the same script but with different input string arguments run in parallel

问题描述

我的问题与

My question is similar with this one, and the difference is that my different arguments are not numbers but strings.

如果我有一个脚本(myscript.R)，它需要两个字符串作为参数:"text-a"，"text-A".我的sbatch外壳程序脚本为:

If I have a script (myscript.R) that takes two strings as arguments: "text-a", "text-A". My shell script for sbatch would be:

#!/bin/bash

#SBATCH -n 1
#SBATCH -c 12
#SBATCH -t 120:00:00
#SBATCH --partition=main
#SBATCH --export=ALL

srun ./myscript.R "text-a" "text-A"

现在我想使用一些不同的输入字符串:

Now I have a few different input strings that I'd like to run with:

first <- c("text-a","text-b","text-c","text-d")
second <- c("text-A","text-B","text-C","text-D")

并且我想使用文本组合来运行myscript.R，例如:

and I want to run myscript.R with combinations of the texts, for example:

srun ./myscript.R "text-a" "text-A"
srun ./myscript.R "text-b" "text-B"
srun ./myscript.R "text-c" "text-C"
srun ./myscript.R "text-d" "text-D"

但是，如果我将它们放在相同的shell脚本中，它们将顺序运行.我只知道当参数为索引时，可以使用 #SBATCH -a 0-10 .如果我想同时提交四个脚本，并且每个脚本都具有完全相同的设置(尤其是每个脚本都需要分配 -c 12 )，该怎么办?

But if I put them in the same shell script, they'll run sequentially. I only know that I can use #SBATCH -a 0-10 when the arguments are index. If I want to submit the four scripts at the same time and each of them with the exact same settings (especially each one need to be assigned -c 12), how can I do that?

谢谢！

推荐答案

您可以将参数值列表存储在数组中，并使用 SLURM_ARRAY_TASK_ID env变量对该数组建立​​索引.

You can store de list of argument values in an array and use the SLURM_ARRAY_TASK_ID env variable to index that array.

#!/bin/bash

#SBATCH -n 1
#SBATCH -c 12
#SBATCH -t 120:00:00
#SBATCH --partition=main
#SBATCH --export=ALL
#SBATCH --array=0-3

A=(text-{a..d}) # This is equivalent to A=(text-a text-b ... text-d)
B=(text-{A..D})

srun ./myscript.R "${A[$SLURM_ARRAY_TASK_ID]}" "${B[$SLURM_ARRAY_TASK_ID]}"

，只需使用 sbatch 提交即可.

这篇关于相同脚本但具有不同输入字符串参数的SLURM抽签作业数组并行运行的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！