在运行下一个命令之前，等待其他会话中的多个并发 powershell 命令完成 [英] Wait for multiple simultaneous powershell commands in other sessions to finish before running next commands

问题描述

我正在尝试获取一个主 powershell 脚本来执行以下操作:

1. 在其他 3 个 powershell 会话中运行命令(它们都运行大约 1 小时 - 我希望它们同时运行，以便它们所做的工作可以同时完成)
2. 等待所有 3 个 其他 powershell 会话完成
3. 继续执行初始 powershell 窗口中的剩余命令

极其简单的例子

我的实际用例与以下类似，除了时间总是变化的，ECHO "hi" 应该只在所有其他 (3) 个命令完成后发生(在这种情况下我们知道它们将需要 10000 秒，但在我的实际用例中，这变化很大).另请注意，不清楚这 3 个命令中的哪一个每次花费的时间最长.

启动 powershell { TIMEOUT 2000 }启动 powershell { 超时 3000 }启动 powershell { 超时 10000 }回声嗨"

我可以看到(这里)我可以在前面放一个 &该命令是为了告诉 powershell 在执行后续命令之前等待它完成.但是，我不知道如何使用 3 个同时命令

解决方案

您确实在寻找 Powershell 后台工作，如Lee Daily 建议.

然而，工作是繁重的，因为每个工作都在自己的流程中运行，这会引入大量开销，并且还可能导致类型保真度丢失(由于涉及 PowerShell 的基于 XML 的序列化基础结构 - 请参阅此答案).

ThreadJob 模块提供了一个基于线程.它带有 PowerShell [Core] v6+，并且在 Windows PowerShell 中可以按需安装，例如
Install-Module ThreadJob -Scope CurrentUser.

您只需调用 Start-ThreadJob 而不是 Start-Job，并使用标准的 *-Job cmdlet 来管理 此类线程作业 - 与您管理常规后台作业的方式相同.

这是一个例子:

$startedAt = [datetime]::UtcNow# 定义作为 [thread] 作业运行的命令.$commands = { $n = 2;开始睡眠 $n;我跑了 $n 秒."},{ $n = 3;开始睡眠 $n;我跑了 $n 秒."},{ $n = 10;开始睡眠 $n;我跑了 $n 秒."}# 启动(线程)作业.# 你可以在这里使用`Start-Job`，但这会更占用资源# 并使脚本运行时间更长.$jobs = $commands |Foreach-Object { Start-ThreadJob $_ }# 等待所有作业完成，然后将它们的输出传递给它# 收到，之后自动清理.$jobs |Receive-Job -Wait -AutoRemoveJob所有作业已完成.总运行时间(以秒为单位).:$(([datetime]::UtcNow - $startedAt).TotalSeconds)"

上面的结果类似于以下内容；请注意，单个命令的输出会在可用时报告，但调用脚本的执行在所有命令都完成之前不会继续:

我跑了 2 秒.我跑了3秒.我跑了10秒.所有作业完成.以秒为单位的总运行时间:10.2504931

注意:在这个简单的例子中，很明显哪个输出来自哪个命令，但更常见的是，来自各种作业的输出将不可预测地交错运行，这使得它难以解释输出 - 请参阅下一节以获得解决方案.

如您所见，后台基于线程的并行执行引入的开销很小 - 整体执行时间仅略长于 10 秒，这是 3 个命令中运行时间最长的运行时间.

如果您改用基于进程的 Start-Job，则整体执行时间可能如下所示，显示引入的大量开销，尤其是您第一次在一个会话:

所有工作都完成了.以秒为单位的总运行时间:18.7502717

也就是说，至少在会话中的第一次调用时，后台并行执行的好处被否定了 - 在这种情况下，执行时间比顺序执行时间长.

虽然同一会话中后续基于进程的后台作业运行速度更快，但开销仍然明显高于基于线程的作业.

<小时>

同步作业输出流

如果你想显示后台命令的输出每个命令，你需要单独收集输出.

注意:在控制台窗口(终端)中，这需要您等到所有命令都完成后才能显示输出(因为无法通过就地更新同时显示多个输出流，至少在常规输出命令).

$startedAt = [datetime]::UtcNow$commands = { $n = 1;开始睡眠 $n;我跑了 $n 秒."},{ $n = 2;开始睡眠 $n;我跑了 $n 秒."},{ $n = 3;开始睡眠 $n;我跑了 $n 秒."}$jobs = $commands |Foreach-Object { Start-ThreadJob $_ }# 等待所有作业完成.$null = 等待工作 $jobs# 为每个作业单独收集输出并打印.foreach ($jobs 中的 $job) {"`n--- {$($job.Command)} 的输出:"接收工作 $job}"`n所有作业已完成.以秒为单位的总运行时间.: $('{0:N2}' -f ([datetime]::UtcNow - $startedAt).TotalSeconds)"

上面将打印如下内容:

<预><代码>--- 输出 { $n = 1;开始睡眠 $n;我跑了 $n 秒."}:我跑了1秒.--- 输出 { $n = 2;开始睡眠 $n;我跑了 $n 秒."}:我跑了2秒.--- 输出 { $n = 3;开始睡眠 $n;我跑了 $n 秒."}:我跑了3秒.所有作业完成.以秒为单位的总运行时间:3.09

<小时>

使用 Start-Process 在单独的窗口中运行命令

在 Windows 上，您可以使用 Start-Process(其别名是 start)在一个新窗口中运行命令，这也是异步默认情况下，即串行启动命令确实并行运行.

在有限的形式中，这允许您实时监控特定于命令的输出，但它带有以下警告:

• 您必须单独手动激活新窗口才能看到生成的输出.

• 输出仅在命令运行时可见；完成后，其窗口会自动关闭，因此您无法事后检查输出.

  • 要解决这个问题，您必须在 PowerShell cmdlet 中使用 Tee-Object 之类的东西，以便同时捕获 文件 中的输出，其中调用者可以稍后检查.

  • 这也是使输出以编程可用的唯一方法，尽管只能作为文本.

• 通过 Start-Process 将 PowerShell 命令传递给 powershell.exe 要求您将命令作为字符串(而不是脚本)传递块)并且有烦人的解析要求，例如需要转义 " 字符.作为 \" (sic) - 见下文.

• 最后一点，使用 Start-Process 也会带来大量的处理开销(尽管很长时间运行的命令可能无关紧要).

$startedAt = [datetime]::UtcNow# 将命令 - 必要 - 定义为 *strings*.# 注意转义嵌入的"chars. as \"的意外需要$commands = '$n = 1;开始睡眠 $n;\"我跑了 $n 秒.\"','$n = 2;开始睡眠 $n;\"我跑了 $n 秒.\"','$n = 3;开始睡眠 $n;\"我跑了 $n 秒.\"'# 使用 `Start-Process` 异步启动命令，# 在每个新窗口中(仅限 Windows).# `-PassThru` 传递一个代表新创建进程的对象.$procs = $commands |ForEach-Object { Start-Process -PassThru powershell -Args '-c', $_ }# 等待所有进程退出.$procs.WaitForExit()"`n所有进程已完成.以秒为单位的总运行时间.: $('{0:N2}' -f ([datetime]::UtcNow - $startedAt).TotalSeconds)"

I am trying to get a master powershell script to do the following:

1. Run commands in 3 other powershell sessions (they all go for about ~1h - I'd like them to run concurrently, so that the jobs they do can all get done at the same time)
2. Wait for all 3 other powershell sessions to finish
3. Continue on with remaining commands in the initial powershell window

Extremely simple example

My real use case is similar to the following, except the times always vary, ECHO "hi" should happen only once all the other (3) commands have finished (in this case we know they'll take 10000 seconds, but in my actual use case this varies a lot). Also note, it's not clear which of the 3 commands will take the longest each time.

start powershell { TIMEOUT 2000 }
start powershell { TIMEOUT 3000 }
start powershell { TIMEOUT 10000 }
ECHO "hi"

I can see (here) that I can put an & in front of the command in order to tell powershell to wait until it's complete before progressing to subsequent commands. However, I do not know how to do so with 3 simultaneous commands

解决方案

You are indeed looking for Powershell background jobs, as Lee Daily advises.

However, jobs are heavy-handed, because each job runs in its own process, which introduces significant overhead, and can also result in loss of type fidelity (due to PowerShell's XML-based serialization infrastructure being involved - see this answer).

The ThreadJob module offers a lightweight alternative based on threads. It comes with PowerShell [Core] v6+ and in Windows PowerShell can be installed on demand with, e.g.,
Install-Module ThreadJob -Scope CurrentUser.

You simply call Start-ThreadJob instead of Start-Job, and use the standard *-Job cmdlets to manage such thread jobs - the same way you'd manage a regular background job.

Here's an example:

$startedAt = [datetime]::UtcNow

# Define the commands to run as [thread] jobs.
$commands = { $n = 2; Start-Sleep $n; "I ran for $n secs." }, 
            { $n = 3; Start-Sleep $n; "I ran for $n secs." }, 
            { $n = 10; Start-Sleep $n; "I ran for $n secs." }

# Start the (thread) jobs.
# You could use `Start-Job` here, but that would be more resource-intensive
# and make the script run considerably longer.
$jobs = $commands | Foreach-Object { Start-ThreadJob $_ }

# Wait until all jobs have completed, passing their output through as it
# is received, and automatically clean up afterwards.
$jobs | Receive-Job -Wait -AutoRemoveJob


"All jobs completed. Total runtime in secs.: $(([datetime]::UtcNow - $startedAt).TotalSeconds)"

The above yields something like the following; note that the individual commands' output is reported as it becomes available, but execution of the calling script doesn't continue until all commands have completed:

I ran for 2 secs.
I ran for 3 secs.
I ran for 10 secs.
All jobs completed. Total runtime in secs.: 10.2504931

Note: In this simple case, it's obvious which output came from which command, but more typically the output from the various jobs will run unpredictably interleaved, which makes it difficult to interpret the output - see the next section for a solution.

As you can see, the overhead introduced for the thread-based parallel execution in the background is minimal - overall execution took only a little longer than 10 seconds, the runtime of the longest-running of the 3 commands.

If you were to use the process-based Start-Job instead, the overall execution time might look something like this, showing the significant overhead introduced, especially the first time you run a background job in a session:

All jobs completed. Total runtime in secs.: 18.7502717

That is, at least on the first invocation in a session, the benefits of parallel execution in the background were negated - execution took longer than sequential execution would have taken in this case.

While subsequent process-based background jobs in the same session run faster, the overhead is still significantly higher than it is for thread-based jobs.

---

Synchronizing the job output streams

If you want show output from the background commands per command, you need to collect output separately.

Note: In a console window (terminal), this requires you to wait until all commands have completed before you can show the output (because there is no way to show multiple output streams simultaneously via in-place updating, at least with the regular output commands).

$startedAt = [datetime]::UtcNow

$commands = { $n = 1; Start-Sleep $n; "I ran for $n secs." }, 
            { $n = 2; Start-Sleep $n; "I ran for $n secs." }, 
            { $n = 3; Start-Sleep $n; "I ran for $n secs." }

$jobs = $commands | Foreach-Object { Start-ThreadJob $_ }

# Wait until all jobs have completed.
$null = Wait-Job $jobs

# Collect the output individually for each job and print it.
foreach ($job in $jobs) {
  "`n--- Output from {$($job.Command)}:"
  Receive-Job $job
} 

"`nAll jobs completed. Total runtime in secs.: $('{0:N2}' -f ([datetime]::UtcNow - $startedAt).TotalSeconds)"

The above will print something like this:

--- Output from { $n = 1; Start-Sleep $n; "I ran for $n secs." }:
I ran for 1 secs.

--- Output from { $n = 2; Start-Sleep $n; "I ran for $n secs." }:
I ran for 2 secs.

--- Output from { $n = 3; Start-Sleep $n; "I ran for $n secs." }:
I ran for 3 secs.

All jobs completed. Total runtime in secs.: 3.09

---

Using Start-Process to run the commands in separate windows

On Windows, you can use Start-Process (whose alias is start) to run commands in a new window, which is also asynchronous by default, i.e., serially launched commands do run in parallel.

In a limited form, this allows you to monitor command-specific output in real time, but it comes with the following caveats:

• You'll have to manually activate the new windows individually to see the output being generated.

• The output is only visible while a command is running; on completion, its window closes automatically, so you can't inspect the output after the fact.

  • To work around that you'd have to use something like Tee-Object in your PowerShell cmdlet in order to also capture output in a file, which the caller could later inspect.

  • This is also the only way to make the output available programmatically, albeit only as text.

• Passing PowerShell commands to powershell.exe via Start-Process requires you to pass your commands as strings (rather than script blocks) and has annoying parsing requirements, such as the need to escape " chars. as \" (sic) - see below.

• Last and not least, using Start-Process also introduces significant processing overhead (though with very long-running commands that may not matter).

$startedAt = [datetime]::UtcNow

# Define the commands - of necessity - as *strings*.
# Note the unexpected need to escape the embedded " chars. as \"
$commands = '$n = 1; Start-Sleep $n; \"I ran for $n secs.\"',
            '$n = 2; Start-Sleep $n; \"I ran for $n secs.\"',
            '$n = 3; Start-Sleep $n; \"I ran for $n secs.\"'

# Use `Start-Process` to launch the commands asynchronously,
# in a new window each (Windows only).
# `-PassThru` passes an object representing the newly created process through.
$procs = $commands | ForEach-Object { Start-Process -PassThru powershell -Args '-c', $_ }

# Wait for all processes to exit.
$procs.WaitForExit()


"`nAll processes completed. Total runtime in secs.: $('{0:N2}' -f ([datetime]::UtcNow - $startedAt).TotalSeconds)"

这篇关于在运行下一个命令之前，等待其他会话中的多个并发 powershell 命令完成的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！