PowerShell的管道爆炸引起的内存使用情况 [英] Powershell piping causes explosive memory usage

问题描述

我目前正在写在PowerShell中的脚本允许在SVN仓库到另一个，而preserving历史上复制一个文件夹。这样的命令的一个例子是：

I am currently writing a script in Powershell that allows copying a folder in a SVN repository to another while preserving the history. An example of such command is:

svnadmin.exe dump $FromRepoPath `
    | svndumpfilter.exe include --drop-empty-revs --renumber-revs --preserve-revprops $Folder `
    | svnadmin.exe load --ignore-uuid $ToRepoPath

这导致PowerShell中非常高的内存使用情况。看来，Powershell的第一执行svnadmin.exe并缓冲SVN管理标准输出，然后执行过滤流和缓冲的输出，并最终执行svnadmin.exe。

This causes an very high memory usage in Powershell. It appears that Powershell first executes svnadmin.exe and buffers the stdout from SVN admin, then executes svndumpfilter and buffers that output and finally executes svnadmin.exe.

我可以解决它通过创建一个单独的批处理文件：

I can work around it by creating a seperate batch file:

@echo off
svnadmin.exe dump %1 | svndumpfilter.exe include --drop-empty-revs --renumber-revs --preserve-revprops %2 | svnadmin.exe load --ignore-uuid %3

然后再从Powershell的调用它：

And then calling it from Powershell:

cmd.exe /c "SvnHelper.cmd $FromRepoPath $Folder $ToRepoPath"

不过，这种感觉就像一个讨厌的和不必要的解决方法。

But this feels like a nasty and unnecessary workaround.

有没有办法告诉Powershell的管道，而不是它的缓冲直接时候直通？

Is there any way to tell Powershell to pass-through directly when piping instead of buffering it?

推荐答案

这不是输出被缓冲，而是投入到任何外部的过程。您可以验证与这样的功能的行为：

It's not the output being buffered, but rather the input to any external process. You can verify the behavior with a function like this:

function Read-Pipeline {
[cmdletbinding()]
param ([Parameter(Mandatory = $true, ValueFromPipeline=$true)] $inp)
    Begin {}
    Process {Write-Verbose $inp ; Return $inp}
    End {}
}

如果您然后运行：

.\LongRunning.exe | Read-Pipeline -Verbose | .\Other.exe

您会看到LongRunning.exe在详细输出，但Other.exe不会运行，直到其管道被关闭。如果你这样做：

You'll see the LongRunning.exe's output in Verbose but Other.exe won't be run until its pipeline is closed. If you do this:

.\LongRunning.exe | Read-Pipeline -Verbose | Write-Host

您会看到详细的交替/控制台输出线，没有缓冲，因为输入不跨越进程边界。

You'll see alternating Verbose / Console output lines with no buffering because inputs aren't crossing process boundaries.

这些都不确实能帮助你。你可以变通的作法是回落到.NET启动流程和手动复制到STDOUT STDIN [1]，但它是小小的奖励了大量的工作。做最简单的事情会通过你的命令CMD，是这样的：

None of that really helps you. You could work around this by falling back to .NET to launch the processes and manually copying STDOUT to STDIN [1], but it's a lot of work for little reward. The simplest thing to do would be pass your commands to CMD, something like:

& cmd.exe "/C svnadmin.exe dump $FromRepoPath ^| svndumpfilter.exe include --drop-empty-revs --renumber-revs --preserve-revprops $Folder ^| svnadmin.exe load --ignore-uuid $ToRepoPath

[1]的http://sushihangover.blogspot.com/2012/01/powershell-piping-standard-output-to.html

这篇关于PowerShell的管道爆炸引起的内存使用情况的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！