通过sbatch传递命令行参数 [英] Pass command line arguments via sbatch

问题描述

假设我有以下简单的bash脚本，希望通过SLURM提交到批处理服务器:

Suppose that I have the following simple bash script which I want to submit to a batch server through SLURM:

#!/bin/bash

#SBATCH -o "outFile"$1".txt"
#SBATCH -e "errFile"$1".txt"

hostname

exit 0

在此脚本中，我只想将 hostname 的输出写在我通过命令行控制其全名的文本文件中，如下所示:

In this script, I simply want to write the output of hostname on a textfile whose full name I control via the command-line, like so:

login-2:jobs$ sbatch -D `pwd` exampleJob.sh 1
Submitted batch job 203775

不幸的是，似乎我的最后一个命令行参数(1)并未通过sbatch进行解析，因为创建的文件没有我要查找的后缀，并且字符串"$ 1"的字面含义为:

Unfortunately, it seems that my last command-line argument (1) is not parsed through sbatch, since the files created do not have the suffix I'm looking for and the string "$1" is interpreted literally:

login-2:jobs$ ls
errFile$1.txt  exampleJob.sh outFile$1.txt

我在 SO 和其他地方，但是我没有任何运气.本质上，我要寻找的是等效于启用Torque的群集中 qsub 实用程序的 -v 开关.

I've looked around places in SO and elsewhere, but I haven't had any luck. Essentially what I'm looking for is the equivalent of the -v switch of the qsub utility in Torque-enabled clusters.

编辑:正如在基础注释线程中提到的那样，我以艰辛的方式解决了我的问题:与其将一个脚本多次提交给批处理服务器，而没有一个脚本，但每个脚本都有不同的命令行参数，我创建了一个主脚本"，该脚本简单地将相同的内容回显并重定向到不同的脚本，每个脚本的内容都由传递的命令行参数更改.然后，我通过 sbatch 将所有这些提交到了批处理服务器.但是，这并不能回答原始问题，因此我不愿意将其添加为我的问题的答案或将该问题标记为已解决.

Edit: As mentioned in the underlying comment thread, I solved my problem the hard way: instead of having one single script that would be submitted several times to the batch server, each with different command line arguments, I created a "master script" that simply echoed and redirected the same content onto different scripts, the content of each being changed by the command line parameter passed. Then I submitted all of those to my batch server through sbatch. However, this does not answer the original question, so I hesitate to add it as an answer to my question or mark this question solved.

推荐答案

以#SBATCH开头的行不会被bash解释，而是被sbatch替换为代码.sbatch选项不支持$ 1 vars(仅％j和其他一些，用％1替换$ 1将不起作用).如果您没有并行运行的不同sbatch进程，则可以尝试

The lines starting with #SBATCH are not interpreted by bash but are replaced with code by sbatch. The sbatch options do not support $1 vars (only %j and some others, replacing $1 by %1 will not work). When you don't have different sbatch processes running in parallel, you could try

#!/bin/bash

touch outFile${1}.txt errFile${1}.txt
rm link_out.sbatch link_err.sbatch 2>/dev/null # remove links from previous runs
ln -s outFile${1}.txt link_out.sbatch
ln -s errFile${1}.txt link_err.sbatch

#SBATCH -o link_out.sbatch
#SBATCH -e link_err.sbatch

hostname
# I do not know about the background processing of sbatch, are the jobs still running
# at this point? When they are, you can not delete the temporary symlinks yet.

exit 0

替代:正如您自己在评论中说的那样，您可以制作一个母本.该脚本可以包含类似

Alternative: As you said in a comment yourself, you could make a masterscript. This script can contain lines like

cat  exampleJob.sh.template | sed -e 's/File.txt/File'$1'.txt/' > exampleJob.sh
# I do not know, is the following needed with sbatch?
chmod +x exampleJob.sh

在模板中，#SBATCH行看起来像

In your template the #SBATCH lines look like

#SBATCH -o "outFile.txt"
#SBATCH -e "errFile.txt"

这篇关于通过sbatch传递命令行参数的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！