PBS脚本-o文件到多个位置 [英] PBS script -o file to multiple locations
问题描述
有时候，当我在PBS群集上运行作业时，我真的很想在两个地方添加作业日志(-o文件). $PBS_O_WORKDIR中的一个用于保持所有内容，而${HOME}/jobOuts/中的一个用于进行greping/awking/etc ...
Sometimes when I run jobs on a PBS cluster, I'd really like the joblog (-o file) in two places. One in the $PBS_O_WORKDIR for keeping everthing together and one ${HOME}/jobOuts/ for greping/awking/etc...
从命令行进行测试可与tee一起使用:
Doing a test from the command line works with tee:
echo "hello" | qsub -o `tee $HOME/out1.o $HOME/out2.o $HOME/out3.o`
但是一旦我尝试将其放入PBS脚本中，如果我将其放入PBS脚本和qsub中，将无法正常工作
But once I try to put this in my PBS script, it does not work if I put it in a PBS script and qsub
####Parameterized PBS Script ####
#PBS -S /bin/bash
#PBS -l nodes=1
#PBS -l walltime=0:01:00
#PBS -j oe
#PBS -o `tee TEE_TEST.o TEE_TEST.${PBS_JOBID}.o`
#PBS -M me@email.com
#PBS -m abe
#PBS -V
cd $PBS_O_WORKDIR
echo `date`
这是qsub和错误:
qsub TEST.pbs
qsub: directive error: -o `tee TEE_TEST.o TEE_TEST.${PBS_JOBID}.o`
我在下面尝试了其他一些操作-没有任何效果.
I tried a few other things below - nothing worked.
-o行(逗号，半冒号和空格):
One -o line (comma, semi colon and space):
#PBS -o ${PBS_JOBNAME}.${PBS_JOBID}.o,${HOME}/jobOuts/${PBS_JOBNAME}.${PBS_JOBID}.o
#PBS -o ${PBS_JOBNAME}.${PBS_JOBID}.o,${HOME}/jobOuts/${PBS_JOBNAME}.${PBS_JOBID}.o
#PBS -o ${PBS_JOBNAME}.${PBS_JOBID}.o ${HOME}/jobOuts/${PBS_JOBNAME}.${PBS_JOBID}.o
和两行:
#PBS -o ${PBS_JOBNAME}.${PBS_JOBID}.o
#PBS -o ${HOME}/jobOuts/${PBS_JOBNAME}.${PBS_JOBID}.o
两个衬板仅采用2nd -o选项，一个衬板不起作用.
The two works liner just takes the 2nd -o option, and the one liners don't work.
有什么建议吗?是否有可能?
Any suggestions? Is it possible?
推荐答案
我研究了 qsub手册页，我认为没有一种方法可以为标准输出和标准错误指定多个输出文件(每个文件).从此页面中获取提示，这是我实现与您的目标相似的方法.您的PBS环境可能有所不同.另外，我不是bash专家，所以可能会有更简洁的方法来实现相同的目的.
I studied a qsub man page and I don't think there is a method to specify more than one output file (each) for standard output and standard error. Taking cues from this page, here is a way I achieved something similar to your goals. Your PBS environment may be a little different. Also, I am not a bash expert, so there may be more concise methods of achieving the same thing.
假设您正在使用默认的-o设置，请在常规作业脚本的末尾添加以下命令:
Assuming you're using the default -o settings, at the end of your regular job script put the commands:
# change to the directory from which this job was submitted
cd $PBS_O_WORKDIR
# the standard output by default will be in file JOBNAME.oJOBID
# on my system, PBS_JOBID has ".machinename" at the end, which needs to be removed
filename=${PBS_JOBNAME}.o${PBS_JOBID%%.*}
echo "${PBS_O_WORKDIR}/copy.pbs $filename" | qsub
这将使用stdin参数启动qsub，告诉它使用要复制的filename参数运行"copy.pbs".我使用的copy.pbs文件是:
This will launch qsub using the stdin arguments telling it to run "copy.pbs" with the filename argument to be copied. The copy.pbs file I used is:
#!/bin/bash
# change to the directory from which this job was submitted
cd $PBS_O_WORKDIR
newfile=${HOME}/jobOuts/$1
cp $1 $newfile
这对我有用，可以将第一个PBS标准输出复制到另一个目录.副作用是，使用qsub运行copy.pbs会创建另外两个输出文件STDIN.e *和STDIN.o *.我认为再次使用qsub是确保完成第一项工作的好主意.为了更加安全，您可以在qsub中使用取决于选项"，例如"-W depend=afterok:$PBS_JOBID copy.pbs $filename" | qsub.但是我没有测试这种方法，就像我说的那样，我不是这方面的专家.
This worked for me to copy the first PBS standard output to another directory. A side effect is that running copy.pbs with qsub creates another two output files, STDIN.e* and STDIN.o*. I figured using qsub again was a good idea to make sure the first job had finished. To be more safe, you could use the "depends on option" with qsub, such as "-W depend=afterok:$PBS_JOBID copy.pbs $filename" | qsub. But I did not test this method and like I said, I'm not an expert in any of this.
这篇关于PBS脚本-o文件到多个位置的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
