PowerShell管道正在执行时没有垃圾回收 [英] No garbage collection while PowerShell pipeline is executing

问题描述

更新：以下错误似乎可以通过PowerShell 5解决。该错误仍然存​​在于3和4中。因此，除非您运行PowerShell 2，否则不要使用管道处理任何大文件。 5，

---

考虑下面的代码片段：

<$ p $ 函数Get-DummyData（）{
for（$ i = 0; $ i -lt 10000000; $ i ++）{
这听起来很大!!我是忍者！更多的话，耶！
}
}

Get-DummyData | Out-Null


这会导致PowerShell内存使用量无法控制地增长。执行 Get-DummyData |后Out-Null 几次，我已经看到PowerShell的内存使用率一路高达4GB。

根据

然而，仅仅因为您调用GC并不意味着私有内存使用将返回到脚本启动之前的值。 GC收集只会收集不再使用的内存。如果存在对某个对象的根引用，则不符合收集（释放）的条件。因此，虽然GC系统通常不会以C / C ++的方式泄漏，但它们可以拥有内存容量，这些内存容量可以保持比对象更长的时间。

对于内存分析器来说，似乎大量的超额内存被具有参数绑定信息的字符串的副本占用： 3.2.0。只要你不使用SingleString开关参数，它应该是有效的。

UPDATE: The following bug seems to be resolved with PowerShell 5. The bug remains in 3 and 4. So don't process any huge files with the pipeline unless you're running PowerShell 2 or 5.

---

Consider the following code snippet:

function Get-DummyData() {
    for ($i = 0; $i -lt 10000000; $i++) {
        "This is freaking huge!! I'm a ninja! More words, yay!"
    }
}

Get-DummyData | Out-Null

This will cause PowerShell memory usage to grow uncontrollably. After executing Get-DummyData | Out-Null a few times, I have seen PowerShell memory usage get all the way up to 4 GB.

According to ANTS Memory Profiler, we have a whole lot of things sitting around in the garbage collector's finalization queue. When I call [GC]::Collect(), the memory goes from 4 GB to a mere 70 MB. So we don't have a memory leak, strictly speaking.

Now, it's not good enough for me to be able to call [GC]::Collect() when I'm finished with a long-lived pipeline operation. I need garbage collection to happen during a pipeline operation. However if I try to invoke [GC]::Collect() while the pipeline is executing...

function Get-DummyData() {
    for ($i = 0; $i -lt 10000000; $i++) {
        "This is freaking huge!! I'm a ninja! More words, yay!"

        if ($i % 1000000 -eq 0) {
            Write-Host "Prompting a garbage collection..."
            [GC]::Collect()
        }
    }
}

Get-DummyData | Out-Null

... the problem remains. Memory usage grows uncontrollably again. I have tried several variations of this, such as adding [GC]::WaitForPendingFinalizers(), Start-Sleep -Seconds 10, etc. I have tried changing garbage collector latency modes and forcing PowerShell to use server garbage collection to no avail. I just can't get the garbage collector to do its thing while the pipeline is executing.

This isn't a problem at all in PowerShell 2.0. It's also interesting to note that $null = Get-DummyData also seems to work without memory issues. So it seems tied to the pipeline, rather than the fact that we're generating tons of strings.

How can I prevent my memory from growing uncontrollably during long pipelines?

Side note:

My Get-DummyData function is only for demonstration purposes. My real-world problem is that I'm unable to read through large files in PowerShell using Get-Content or Import-Csv. No, I'm not storing the contents of these files in variables. I'm strictly using the pipeline like I'm supposed to. Get-Content .\super-huge-file.txt | Out-Null produces the same problem.

解决方案

A couple of things to point out here. First, GC calls do work in the pipeline. Here's a pipeline script that only invokes the GC:

1..10 | Foreach {[System.GC]::Collect()}

Here's the perfmon graph of GCs during the time the script ran:

However, just because you invoke the GC it doesn't mean the private memory usage will return to the value you had before your script started. A GC collect will only collect memory that is no longer used. If there is a rooted reference to an object, it is not eligible to be collected (freed). So while GC systems typically don't leak in the C/C++ sense, they can have memory hoards that hold onto objects longer than perhaps they should.

In looking at this with a memory profiler it seems the bulk of the excess memory is taken up by a copy of the string with parameter binding info:

The root for these strings look like this:

I wonder if there is some logging feature that is causing PowerShell to hang onto a string-ized form pipeline bound objects?

BTW in this specific case, it is much more memory efficient to assign to $null to ignore the output:

$null = GetDummyData

Also, if you need to simply edit a file, check out the Edit-File command in the PowerShell Community Extensions 3.2.0. It should be memory efficient as long as you don't use the SingleString switch parameter.

这篇关于PowerShell管道正在执行时没有垃圾回收的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！