在 bash 中跨多个内核运行循环进程 [英] Run a looped process in bash across multiple cores

问题描述

我有一个包含以下循环的 shell 脚本.

I have a shell script that contains the following loop.

i=0  
upperlimit=$verylargevariable  
do  
   complexstuff RunManager file $i  
   i= 'expr $i +1'  
done

此脚本在四核机器上运行，根据 top 的说法，在执行循环的一次迭代时使用了每个内核的大约 15%.我想将它分布在四个核心上，以便循环的每次迭代执行 complexstuff 四次，每个核心上一次，这样资源的使用效率就会更高.我们谈论的是目前需要几个小时的计算，因此效率不仅仅是这里的好习惯.(每次迭代的输出显然与前一次无关.)

This script runs on a quad core machine, and according to top, uses about 15% of each core while executing one iteration of the loop. I'd like to distribute it across the four cores so that each iteration of the loop does complexstuff four times, one on each core, so the resources will be used more efficiently. We're talking about computation that currently takes several hours so efficiency is more than just good practice here. (The output of each iteration is obviously independent of the previous one.)

PS:主机是运行 Cent-OS 的服务器，如果有帮助的话.

PS: Host is a server running Cent-OS, if that helps.

推荐答案

除了 Ole Tange 解决方案(看起来很棒)，如果你的计算有非常相似的持续时间，你可以尝试这样的事情:

Apart the Ole Tange solution (that looks great), if your computations have pretty similar durations, you can try something like this :

i=0  
upperlimit=$verylargevariable  
do  
   complexstuff RunManager file $i &
   i= 'expr $i + 1'
   complexstuff RunManager file $i &
   i= 'expr $i + 1'
   complexstuff RunManager file $i &
   i= 'expr $i + 1'
   complexstuff RunManager file $i &
   i= 'expr $i + 1'
   wait
done

这样，在循环的每次运行中，您将创建 4 个 bash 子进程来启动您的计算(并且由于系统很棒，它会将它们分派到不同的内核上).如果 4 个进程不足以烧毁所有 CPU，请增加每个循环上创建的进程数.

This way, on each run of the loop, you will create 4 bash subprocesses that will launch your computations (and as system is great, it will dispatch them on the different cores). If with 4 processes it is not enough to burn all your cpus, raise the number of processes created on each loop.

这篇关于在 bash 中跨多个内核运行循环进程的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！