Bash:将来自多个xargs并行进程的stdout记录到单独的日志文件中 [英] Bash: Logging stdout from multiple xargs parallel processes to separate log files

问题描述

我正在处理由xargs产生的多个并行进程的文本文件.我还需要将每个进程的标准输出捕获到单独的日志文件中.下面是一个示例，其中每个进程的输出被交错到一个文件中，而不是我想要的文件.

I am processing a text file with multiple parallel processes spawned by xargs. I also need to capture the stdout from each process into a separate log file. Below is an example where the output from each process is interleaved into a single file -- not what I want.

理想情况下，每个日志文件都应按文件行号编号，即logfile-1，logfile-2等.

Ideally, each logfile should be numbered by the file line number, that is, logfile-1, logfile-2, etc.

cat inputfile.txt | xargs -n 1 -P 8 ./myScript.sh | tee logfile

如果可能的话，最好避免使用外部包装脚本，但是如果有一种方法可以将myScript与here文档包装在一起，那将是可行的.

It would be nice to avoid an external wrapper script if possible, but if there is a way to wrap myScript with a here document, that would work.

推荐答案

尝试一下:

nl inputfile.txt | xargs -n 2 -P 8 sh -c './myScript.sh "$1" > logfile-$0'

这假定 inputfile.txt 中的每个参数都位于其自己的行上，并且不包含空格. nl 命令为每行编号，该代码将每个参数与唯一编号配对. xargs 命令同时接受两个参数，第一个是行号，第二个是来自 inputfile.txt 的对应行，并将它们传递给 sh . sh 命令使用参数分别生成输出文件名和 myScript.sh 的参数.

This assumes each argument in inputfile.txt is on its own line and contains no spaces. The nl command numbers each line, which pairs each argument with a unique number. The xargs commands takes two arguments at time, the first the line number, the second the corresponding line from inputfile.txt, and passes them to sh. The sh command uses the arguments to generate the output file name and the argument to myScript.sh respectively.

这篇关于Bash:将来自多个xargs并行进程的stdout记录到单独的日志文件中的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！