通过作业完成的Powershell节流多线程作业 [英] Powershell Throttle Multi thread jobs via job completion

问题描述

我发现的所有孩子都使用预定义的睡眠时间来限制工作. 我需要节流阀等到一项工作完成后才能开始新的工作. 一次只能运行4个作业.

All the tuts I have found use a pre defined sleep time to throttle jobs. I need the throttle to wait until a job is completed before starting a new one. Only 4 jobs can be running at one time.

因此该脚本将运行4，当前暂停10秒，然后运行其余部分. 我想要的是该脚本一次只允许运行4个作业，并在完成一项作业后启动一个新作业.

So The script will run up 4 and currently pauses for 10 seconds then runs up the rest. What I want is for the script to only allow 4 jobs to be running at one time and as a job is completed a new one is kicked off.

作业通过服务器名称列表进行初始化.

Jobs are initialised via a list of servers names.

可以存档吗?

$servers = Get-Content "C:\temp\flashfilestore\serverlist.txt"

$scriptBlock = { #DO STUFF }


$MaxThreads = 4

foreach($server in $servers) {
     Start-Job -ScriptBlock $scriptBlock -argumentlist  $server 
     While($(Get-Job -State 'Running').Count -ge $MaxThreads) {
          sleep 10 #Need this to wait until a job is complete and kick off a new one.
     }
}
Get-Job | Wait-Job | Receive-Job

推荐答案

为避免发明轮子，我建议使用以下一种 现有工具.

In order to avoid inventing a wheel I would recommend to use one of the existing tools.

其中之一是脚本 调用Parallel.ps1 . 它是用PowerShell编写的，您可以看到它是如何直接实现的.它是 易于获取，不需要任何安装即可使用.

One of them is the script Invoke-Parallel.ps1. It is written in PowerShell, you can see how it is implemented directly. It is easy to get and it does not require any installation for using it.

另一个是模块 SplitPipeline . 因为它是用C#编写的，所以它可能工作得更快.它还涵盖了更多用途 情况，例如缓慢或无限的输入，请使用初始化和清除脚本.

Another one is the module SplitPipeline. It may work faster because it is written in C#. It also covers some more use cases, for example slow or infinite input, use of initialization and cleanup scripts.

在后一种情况下，具有4个并行管道的代码将为

In the latter case the code with 4 parallel pipelines will be

$servers | Split-Pipeline -Count 4 {process{ <# DO STUFF on $_ #> }}

这篇关于通过作业完成的Powershell节流多线程作业的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！