如何我在bash脚本使用并行编程/多线程？ [英] How do I use parallel programming/multi threading in my bash script?

问题描述

这是我的脚本：

#!/bin/bash
#script to loop through directories to merge fastq files
sourcedir=/path/to/source
destdir=/path/to/dest

for f in $sourcedir/*
do
    fbase=$(basename "$f")
    echo "Inside $fbase"
    zcat $f/*R1*.fastq.gz | gzip > $destdir/"$fbase"_R1.fastq.gz
    zcat $f/*R2*.fastq.gz | gzip > $destdir/"$fbase"_R2.fastq.gz
done

下面有目录来源约30子目录。每个子目录具有一定的 R1 的.fastq.gz文件和 R2 的.fastq.gz，我要合并成一个R1.fastq.gz和R2.fastq.gz文件，然后保存文件合并到目标目录。我的code工作正常，但我需要加快速度，因为数据量。我只是想知道有没有什么办法，我可以实现我的剧本多线程编程？我如何运行我的脚本，使多个作业并行运行？新来bash脚本编程，所以任何帮助将是AP preciated。

Here there are about 30 sub-directories in the directory 'source'. Each sub-directory has certain R1.fastq.gz files and R2.fastq.gz that I want to merge into one R1.fastq.gz and R2.fastq.gz file, then save the merged file to the destination directory. My code works fine but I need to speed it up because of the amount of data. I just want to know is there any way I can implement multi threaded programming in my script? How can I run my script so that multiple jobs run in parallel? New to bash scripting, so any help would be appreciated.

推荐答案

最简单的方法是在背景中加入＆放大器来执行这些命令; 来的结束命令：

The simplest way is to execute the commands in the background, by adding & to the end of the command:

#!/bin/bash
#script to loop through directories to merge fastq files
sourcedir=/path/to/source
destdir=/path/to/dest

for f in $sourcedir/*
do
    fbase=$(basename "$f")
    echo "Inside $fbase"
    zcat $f/*R1*.fastq.gz | gzip > $destdir/"$fbase"_R1.fastq.gz &
    zcat $f/*R2*.fastq.gz | gzip > $destdir/"$fbase"_R2.fastq.gz &
done

从 bash的手动：

如果一个命令是由控制操作符'和;'，外壳异步执行的命令在子shell。这被称为在后台执行命令。 shell不等待命令完成，返回状态是0（真）。当作业控制处于不活动状态（参见作业控制），异步命令的标准输入，在没有任何明确的重定向的，是从/ dev / null的重定向。

If a command is terminated by the control operator ‘&’, the shell executes the command asynchronously in a subshell. This is known as executing the command in the background. The shell does not wait for the command to finish, and the return status is 0 (true). When job control is not active (see Job Control), the standard input for asynchronous commands, in the absence of any explicit redirections, is redirected from /dev/null.

这篇关于如何我在bash脚本使用并行编程/多线程？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！