在运行下一个命令之前等待其他会话中的多个同时执行的 powershell 命令完成 [英] Wait for multiple simultaneous powershell commands in other sessions to finish before running next commands

问题描述

我正在尝试获取一个主 powershell 脚本来执行以下操作:

I am trying to get a master powershell script to do the following:

1. 在其他 3 个 powershell 会话中运行命令(它们都运行大约 1 小时 - 我希望它们同时运行，以便它们所做的工作可以同时完成)
2. 等待所有 3 个 其他 powershell 会话完成
3. 继续执行初始 powershell 窗口中的剩余命令

1. Run commands in 3 other powershell sessions (they all go for about ~1h - I'd like them to run concurrently, so that the jobs they do can all get done at the same time)
2. Wait for all 3 other powershell sessions to finish
3. Continue on with remaining commands in the initial powershell window

极其简单的例子

我的实际用例与以下类似，除了时间总是变化的，ECHO "hi" 应该只在所有其他 (3) 个命令完成后发生(在这种情况下我们知道它们将需要 10000 秒，但在我的实际用例中，这变化很大).另请注意，不清楚这 3 个命令中的哪一个每次花费的时间最长.

Extremely simple example

My real use case is similar to the following, except the times always vary, ECHO "hi" should happen only once all the other (3) commands have finished (in this case we know they'll take 10000 seconds, but in my actual use case this varies a lot). Also note, it's not clear which of the 3 commands will take the longest each time.

start powershell { TIMEOUT 2000 }
start powershell { TIMEOUT 3000 }
start powershell { TIMEOUT 10000 }
ECHO "hi"

我可以看到(这里)我可以在前面放一个 &该命令是为了告诉 powershell 在执行后续命令之前等待它完成.但是，我不知道如何使用 3 个同时执行的命令

I can see (here) that I can put an & in front of the command in order to tell powershell to wait until it's complete before progressing to subsequent commands. However, I do not know how to do so with 3 simultaneous commands

推荐答案

您确实在寻找 Powershell 后台工作，如Lee Daily 建议.

You are indeed looking for Powershell background jobs, as Lee Daily advises.

然而，作业是繁重的，因为每个作业都在自己的流程中运行，这会引入大量开销，并且还可能导致类型保真度丢失(由于涉及 PowerShell 的基于 XML 的序列化基础结构 - 请参阅此答案).

However, jobs are heavy-handed, because each job runs in its own process, which introduces significant overhead, and can also result in loss of type fidelity (due to PowerShell's XML-based serialization infrastructure being involved - see this answer).

ThreadJob 模块 提供了一个基于线程.它随 PowerShell [Core] v6+ 一起提供，在 Windows PowerShell 中可以按需安装，例如
Install-Module ThreadJob -Scope CurrentUser.^[1]

您只需调用 Start-ThreadJob 而不是 Start-Job，并使用标准的 *-Job cmdlet 来管理 此类线程作业 - 与您管理常规后台作业的方式相同.

You simply call Start-ThreadJob instead of Start-Job, and use the standard *-Job cmdlets to manage such thread jobs - the same way you'd manage a regular background job.

这是一个例子:

$startedAt = [datetime]::UtcNow

# Define the commands to run as [thread] jobs.
$commands = { $n = 2; Start-Sleep $n; "I ran for $n secs." }, 
            { $n = 3; Start-Sleep $n; "I ran for $n secs." }, 
            { $n = 10; Start-Sleep $n; "I ran for $n secs." }

# Start the (thread) jobs.
# You could use `Start-Job` here, but that would be more resource-intensive
# and make the script run considerably longer.
$jobs = $commands | Foreach-Object { Start-ThreadJob $_ }

# Wait until all jobs have completed, passing their output through as it
# is received, and automatically clean up afterwards.
$jobs | Receive-Job -Wait -AutoRemoveJob


"All jobs completed. Total runtime in secs.: $(([datetime]::UtcNow - $startedAt).TotalSeconds)"

上面的结果类似于以下内容；请注意，单个命令的输出会在可用时报告，但调用脚本的执行在所有命令都完成之前不会继续:

The above yields something like the following; note that the individual commands' output is reported as it becomes available, but execution of the calling script doesn't continue until all commands have completed:

I ran for 2 secs.
I ran for 3 secs.
I ran for 10 secs.
All jobs completed. Total runtime in secs.: 10.2504931

注意:在这个简单的例子中，很明显哪个输出来自哪个命令，但更常见的是，来自各种作业的输出将不可预测地交错运行，这使得难以解释输出 - 请参阅下一节以获得解决方案.

Note: In this simple case, it's obvious which output came from which command, but more typically the output from the various jobs will run unpredictably interleaved, which makes it difficult to interpret the output - see the next section for a solution.

如您所见，后台基于线程的并行执行引入的开销很小 - 整体执行时间仅略长于 10 秒，这是 3 个命令中运行时间最长的运行时间.

As you can see, the overhead introduced for the thread-based parallel execution in the background is minimal - overall execution took only a little longer than 10 seconds, the runtime of the longest-running of the 3 commands.

如果您改用基于进程的 Start-Job，则整体执行时间可能如下所示，显示引入的大量开销，尤其是您第一次在一个会话:

If you were to use the process-based Start-Job instead, the overall execution time might look something like this, showing the significant overhead introduced, especially the first time you run a background job in a session:

All jobs completed. Total runtime in secs.: 18.7502717

也就是说，至少在会话中的第一次调用时，后台并行执行的好处被否定了 - 在这种情况下，执行时间比顺序执行时间长.

That is, at least on the first invocation in a session, the benefits of parallel execution in the background were negated - execution took longer than sequential execution would have taken in this case.

虽然同一会话中后续基于进程的后台作业运行得更快，但开销仍然明显高于基于线程的作业.

While subsequent process-based background jobs in the same session run faster, the overhead is still significantly higher than it is for thread-based jobs.

如果你想显示后台命令的输出每个命令，你需要单独收集输出.

If you want show output from the background commands per command, you need to collect output separately.

注意:在控制台窗口(终端)中，这需要您等到所有命令都完成后才能显示输出(因为无法通过就地更新同时显示多个输出流，至少在常规输出命令).

Note: In a console window (terminal), this requires you to wait until all commands have completed before you can show the output (because there is no way to show multiple output streams simultaneously via in-place updating, at least with the regular output commands).

$startedAt = [datetime]::UtcNow

$commands = { $n = 1; Start-Sleep $n; "I ran for $n secs." }, 
            { $n = 2; Start-Sleep $n; "I ran for $n secs." }, 
            { $n = 3; Start-Sleep $n; "I ran for $n secs." }

$jobs = $commands | Foreach-Object { Start-ThreadJob $_ }

# Wait until all jobs have completed.
$null = Wait-Job $jobs

# Collect the output individually for each job and print it.
foreach ($job in $jobs) {
  "`n--- Output from {$($job.Command)}:"
  Receive-Job $job
} 

"`nAll jobs completed. Total runtime in secs.: $('{0:N2}' -f ([datetime]::UtcNow - $startedAt).TotalSeconds)"

上面将打印如下内容:

--- Output from { $n = 1; Start-Sleep $n; "I ran for $n secs." }:
I ran for 1 secs.

--- Output from { $n = 2; Start-Sleep $n; "I ran for $n secs." }:
I ran for 2 secs.

--- Output from { $n = 3; Start-Sleep $n; "I ran for $n secs." }:
I ran for 3 secs.

All jobs completed. Total runtime in secs.: 3.09

---

使用 Start-Process 在单独的窗口中运行命令

在 Windows 上，您可以使用 Start-Process(其别名是 start)在一个新窗口中运行命令，这也是异步默认情况下，即串行启动命令确实并行运行.

---

Using Start-Process to run the commands in separate windows

On Windows, you can use Start-Process (whose alias is start) to run commands in a new window, which is also asynchronous by default, i.e., serially launched commands do run in parallel.

在有限的形式中，这允许您实时监控特定于命令的输出，但它带有以下警告:

In a limited form, this allows you to monitor command-specific output in real time, but it comes with the following caveats:

• 您必须单独手动激活新窗口才能查看生成的输出.

• You'll have to manually activate the new windows individually to see the output being generated.

输出仅在命令运行时可见；完成后，其窗口会自动关闭，因此您无法事后检查输出.

The output is only visible while a command is running; on completion, its window closes automatically, so you can't inspect the output after the fact.

• 要解决这个问题，您必须在 PowerShell cmdlet 中使用 Tee-Object 之类的东西，以便同时捕获 文件 中的输出，其中调用者可以稍后检查.

• To work around that you'd have to use something like Tee-Object in your PowerShell cmdlet in order to also capture output in a file, which the caller could later inspect.

这也是使输出以编程可用的唯一方法，尽管只能作为文本.

This is also the only way to make the output available programmatically, albeit only as text.

通过 Start-Process 将 PowerShell 命令传递给 powershell.exe 要求您将命令作为字符串(而不是脚本)传递块)并且有烦人的解析要求，例如需要转义 " 字符.作为 " (sic) - 见下文.

Passing PowerShell commands to powershell.exe via Start-Process requires you to pass your commands as strings (rather than script blocks) and has annoying parsing requirements, such as the need to escape " chars. as " (sic) - see below.

最后一点，使用 Start-Process 还会引入大量的处理开销(尽管运行时间很长的命令可能无关紧要).

Last and not least, using Start-Process also introduces significant processing overhead (though with very long-running commands that may not matter).

$startedAt = [datetime]::UtcNow

# Define the commands - of necessity - as *strings*.
# Note the unexpected need to escape the embedded " chars. as "
$commands = '$n = 1; Start-Sleep $n; "I ran for $n secs."',
            '$n = 2; Start-Sleep $n; "I ran for $n secs."',
            '$n = 3; Start-Sleep $n; "I ran for $n secs."'

# Use `Start-Process` to launch the commands asynchronously,
# in a new window each (Windows only).
# `-PassThru` passes an object representing the newly created process through.
$procs = $commands | ForEach-Object { Start-Process -PassThru powershell -Args '-c', $_ }

# Wait for all processes to exit.
$procs.WaitForExit()


"`nAll processes completed. Total runtime in secs.: $('{0:N2}' -f ([datetime]::UtcNow - $startedAt).TotalSeconds)"

---

^{[1] 在 Windows PowerShell v3 和 v4 中，Install-Module 默认不可用，因为这些版本不附带 PowerShellGet 模块.但是，此模块可以按需安装，详见 安装PowerShellGet}

---

^{[1] In Windows PowerShell v3 and v4, Install-Module isn't available by default, because these versions do not come with the PowerShellGet module. However, this module can be installed on demand, as detailed in Installing PowerShellGet}

这篇关于在运行下一个命令之前等待其他会话中的多个同时执行的 powershell 命令完成的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！