通过LSF批处理作业中的grep -v重定向stderr [英] Redirect stderr through grep -v in LSF batch job

问题描述

我正在使用一个库，生成一整吨的输出到stderr(并且实际上没有办法直接在代码中抑制输出；这是ROOT的Minuit2最小化器，它没有抑制方法.输出).我正在通过LSF系统运行批处理作业，错误输出文件太大，以至于超出了我的磁盘配额.没关系.

I'm using a library that generates a whole ton of output to stderr (and there is really no way to suppress the output directly in the code; it is ROOT's Minuit2 minimizer which is known for not having a way to suppress the output). I'm running batch jobs through the LSF system, and the error output files are so big that they exceed my disk quota. Erk.

当我在shell上本地运行时，我会这样做:

When I run locally on a shell, I do:

python main.py 2> >( grep -v Minuit2 2>&1 )

如在此处所示，抑制输出. 这很好用，但是不幸的是，当我在LSF上运行时，我似乎无法获得它或它的任何变体.我认为这是由于LSF没有产生必要的subshel​​l，但尚不清楚.

to suppress the output, as is done here. This works great, but unfortunately I can't seem to get that or any variation of it to work when running on LSF. I think this is due to LSF not spawning the necessary subshell, but it's not clear.

我通过向LSF提交提交脚本来批量运行.相关行是:

I run on batch by passing LSF a submit script. The relevant line is:

python main.py $INPUT_FILE

除了前面提到的巨大错误文件问题之外，效果很好.

which works great, aside from the aforementioned problem of gigantic error files.

当我尝试将该行更改为

python main.py $INPUT_FILE 2> >( grep -v Minuit2 2>&1 )

我最终以

./singleSubmit.sh: line 16: syntax error near unexpected token `>'
./singleSubmit.sh: line 16: `python $MAIN $1 2> >( grep -v Minuit2 2>&1 )'

错误日志文件中的

.

in the error log file.

有什么主意我能完成我想要的事情，或者为什么这不起作用?

Any idea how I could accomplish what I want, or why this is not working?

一吨！

推荐答案

您使用的语法在bash中有效，而不在csh/tcsh中有效.尝试将提交脚本的第一行更改为

The syntax you're using works in bash, not in csh/tcsh. Try changing the first line of your submission script to

#!/bin/bash

这篇关于通过LSF批处理作业中的grep -v重定向stderr的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！