如何在Platform LSF中的bsub数组作业参数中使用LSB_JOBINDEX? [英] How to use LSB_JOBINDEX in bsub array job arguments in Platform LSF?

问题描述

我想将LSB_JOBINDEX作为参数传递给脚本，而不是使用环境变量.

这使我的脚本与LSF无关，并且避免创建使用环境变量的帮助程序脚本.

但是，我无法在参数中使用LSB_JOBINDEX:它仅作为初始命令字符串的一部分.

例如，在bash shell中，我使用测试命令:

bsub -J 'myjobname[1-4]' -o bsub%I.log \
  'echo $LSB_JOBINDEX' \
  '$LSB_JOBINDEX' \
  \$LSB_JOBINDEX \
  '$LSB_JOBINDEX' \
  "\$LSB_JOBINDEX"

和说bsub2.log的输出是:

2 $LSB_JOBINDEX $LSB_JOBINDEX $LSB_JOBINDEX $LSB_JOBINDEX

因此，在这种情况下，只有第一个$LSB_JOBINDEX被扩展，而随后的任何一个都没有扩展.

但是在此示例中，我宁愿不像'echo $LSB_JOBINDEX'那样将整个命令作为单个大字符串传递.我宁愿像在常规bash命令中一样使用单独的参数.

我也尝试过使用%I，但是它仅适用于-o和相关的bsub选项，不适用于命令本身.

相关:在LSF作业数组中引用作业索引

在LSF 10.1.0中测试.相关文档: https://www.ibm.com/support/knowledgecenter/zh-CN/SSWRJV_10.1.0/lsf_admin/job_array_cl_args.html

解决方案

如果自变量以$开头，bsub将在自变量周围添加单引号.例如.如果bsub命令行是

bsub command -a $ARG1 -b $ARG2

然后bsub会将引号添加到第二个和第四个参数的参数中.命令存储如下

command -a '$ARG1' -b '$ARG2'

防止这种情况的一种方法是将命令放在脚本中.像这样:

$ cat cmd
echo $LSB_JOBINDEX
echo "line 2"
echo $LSB_JOBINDEX

然后像这样运行您的工作:

$ bsub -I < cmd
Job <2669> is submitted to default queue <normal>.
<<Waiting for dispatch ...>>
<<Starting on hostA>>
0
line 2
0

请注意，不需要-I.只是这样，您就可以在bsub的stdout上看到作业输出.

编辑

好.看起来像这样.但这不是一个严肃的答案，因为它是如此丑陋.问题是，如果参数以$开头，则bsub会将参数括在单引号中.因此，策略是找到某种方法来确保参数中的第一个字符不是$.一种方法是将$以外的任何其他字符用作参数的第一个字符.在其后跟一个退格字面量，后跟$.请注意，它必须是实际的退格字符，而不是^后跟H.使用ctrl-v后跟ctrl-h可以将文字附加到命令行.

$ bsub -I echo "x^H\$LSB_JOBINDEX" "x^H\$LSB_JOBINDEX"
Job <2686> is submitted to default queue <normal>.
<<Waiting for dispatch ...>>
<<Starting on hostA>>
0 0

EDIT2

标签文字也可以使用.没那么好.

$ bsub -I echo "       \$LSB_JOBINDEX" "       \$LSB_JOBINDEX"
Job <2687> is submitted to default queue <normal>.
<<Waiting for dispatch ...>>
<<Starting on hostA>>
0 0

I would like to pass LSB_JOBINDEX to as an argument to my script instead of using an environment variable.

This makes my script more LSF agnostic and avoids creating a helper script that uses the environment variable.

However, I was not able to use LSB_JOBINDEX in arguments: it only works as part of the initial command string.

For example, from a bash shell, I use the test command:

bsub -J 'myjobname[1-4]' -o bsub%I.log \
  'echo $LSB_JOBINDEX' \
  '$LSB_JOBINDEX' \
  \$LSB_JOBINDEX \
  '$LSB_JOBINDEX' \
  "\$LSB_JOBINDEX"

and the output of say bsub2.log is:

2 $LSB_JOBINDEX $LSB_JOBINDEX $LSB_JOBINDEX $LSB_JOBINDEX

So in this case, only the first $LSB_JOBINDEX got expanded, but not any of the following ones.

But I would rather not pass the entire command as a single huge string as the 'echo $LSB_JOBINDEX' in this example. I would prefer to just use separate arguments as in a regular bash command.

I've also tried to play around with %I but it only works for -o and related bsub options, not for the command itself.

Related: Referencing job index in LSF job array

Tested in LSF 10.1.0. Related documentation: https://www.ibm.com/support/knowledgecenter/en/SSWRJV_10.1.0/lsf_admin/job_array_cl_args.html

解决方案

bsub will add single quotes around the arguments if the argument starts with $. For example. If the bsub command line is

bsub command -a $ARG1 -b $ARG2

Then bsub will add quotes to the arguments to the 2nd and 4th parameters. The command is stored like this

command -a '$ARG1' -b '$ARG2'

One way to prevent this is to put the commands in a script. Like this:

$ cat cmd
echo $LSB_JOBINDEX
echo "line 2"
echo $LSB_JOBINDEX

Then run your job like this:

$ bsub -I < cmd
Job <2669> is submitted to default queue <normal>.
<<Waiting for dispatch ...>>
<<Starting on hostA>>
0
line 2
0

Note that the -I is not needed. Its just so you can see the job output on the bsub's stdout.

EDIT

OK. Looks like this works. But its not really a serious answer since it's so ugly. The thing is that bsub will surround the argument with single quotes if the argument starts with $. So the strategy is to find some way to make sure that the first character in the argument isn't a $. One way is to put any character other than $ as the first character of the argument. Follow it by a backspace literal, followed by the $. Note that it needs to be the actual backspace character, not ^ followed by H. Use ctrl-v followed by a ctrl-h to get the literal appended to the command line.

$ bsub -I echo "x^H\$LSB_JOBINDEX" "x^H\$LSB_JOBINDEX"
Job <2686> is submitted to default queue <normal>.
<<Waiting for dispatch ...>>
<<Starting on hostA>>
0 0

EDIT2

A tab literal also works. Not that its much better.

$ bsub -I echo "       \$LSB_JOBINDEX" "       \$LSB_JOBINDEX"
Job <2687> is submitted to default queue <normal>.
<<Waiting for dispatch ...>>
<<Starting on hostA>>
0 0

这篇关于如何在Platform LSF中的bsub数组作业参数中使用LSB_JOBINDEX?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！