如何将当前项目返回到管道? [英] How to return current item back to the pipeline?

问题描述

我目前正在导入 csv 文件并批量加载到 sql 表.

Im currently importing csv files and bulk loading to sql table.

使用此代码

$CSVDataTable = Import-Csv $csvFile | % -begin {$i=0} -process { Write-Progress -activity "Importing file" -currentOperation "Reading line $i" -PercentComplete -1; $i++; return $_ } | Out-DataTable

我能够显示进度，但我想对其进行优化，我发现的一项建议是使用 StreamReader.

i am able to show the progress, but I'd like to optimize it and one recommendation Ive found is utilizing StreamReader.

所以我尝试了以下方法:

so ive tried the following:

[int]$LinesInFile = 0
$reader = New-Object IO.StreamReader $csvFile
$line = $reader.ReadLine()
while($reader.ReadLine() -ne $null) { $LinesInFile++ }

$CSVDataTable = 0..($LinesInFile-1) | foreach {
    $percent = ($_/$LinesInFile)*100
    Write-Progress -Activity 'Importing from CSV' -Status "$percent % Complete" -CurrentOperation "Importing row # $($_+1)" -PercentComplete $percent;
    return $reader[$_]
} | Import-Csv $csvFile | Out-DataTable

错误(由于return $reader[$_]):

Import-Csv : The input object cannot be bound to any parameters for the command either because the command does not take pipeline input or the input and its
properties do not match any of the parameters that take pipeline input.

推荐答案

唯一有意义的管道到 Import-Csv 是 System.IO.FileInfo 实例表示 CSV 文件，例如 Get-ChildItem 的输出(顺便说一句，由于 Windows PowerShell 中的错误而损坏，因为已在 PowerShell [Core] v6+ 中修复)).

The only thing that makes sense to pipe to Import-Csv are System.IO.FileInfo instances representing CSV files, such as output by Get-ChildItem (which, as an aside, is broken due to a bug in Windows PowerShell, since fixed in PowerShell [Core] v6+).

如果您想报告 Import-Csv 调用的进度，请将 ForEach-Object 命令放在之后，其中您可以发出进度消息，然后通过以下方式传递 Import-Csv 的输出对象 ($_):

If you want to report on the progress of an Import-Csv call, put a ForEach-Object command after it, in which you can emit the progress message and then pass Import-Csv's output object ($_) through:

# Count the data rows in the input CSV file.
$rowCount = 0
switch -File $csvFile { default { ++$rowCount } }
--$rowCount # subtract 1 from the line count to account for the header row.

Import-Csv $csvFile | ForEach-Object -Begin { $i = 0 } {

  $percent = '{0:N1}' -f (++$i / $rowCount * 100)
  Write-Progress -Activity 'Importing from CSV' -Status "$percent % Complete" -CurrentOperation "Importing row # $i" -PercentComplete $percent

  $_  # pass the object from Import-Csv through.

} | Out-DataTable

# Hide the progress bar now.
# (Otherwise it would linger until the script as a whole completes.)
Write-Progress '(unused))' -Completed

---

值得注意的是，每个对象 Write-Progress 调用会显着减慢执行速度(并且预先计算行数也有其成本，即使只是通过 ForEach-Object 调用传递对象).

---

It's worth noting that per-object Write-Progress calls significantly slow down execution (and counting the number of lines up front has its cost too, as does even just passing objects through via a ForEach-Object call).

缓解速度缓慢的一种简单方法是仅对每 N 个对象使用 Write-Progress，例如每个 100 个对象在下面的例子:

A simple way to mitigate the slowdown is to only use Write-Progress for every N objects, such as every 100 objects in the following example:

# Count the data rows in the input CSV file.
$rowCount = 0
switch -File $csvFile { default { ++$rowCount } }
--$rowCount # subtract 1 from the line count to account for the header row.

Import-Csv $csvFile | ForEach-Object -Begin { $i = 0 } {

  if (++$i % 100 -eq 1 -or $i -eq $rowCount) {
    $percent = '{0:N1}' -f ($i / $rowCount * 100)
    Write-Progress -Activity 'Importing from CSV' -Status "$percent % Complete" -CurrentOperation "Importing row # $i" -PercentComplete $percent
  }

  $_  # pass the object from Import-Csv through.

} | Out-DataTable

# Hide the progress bar now.
# (Otherwise it would linger until the script as a whole completes.)
Write-Progress '(unused))' -Completed

注意:如果 Out-DataTable 在将最后一个对象传递给它后需要大量的处理时间，您可以填充行数以基于百分比显示估计额外的时间，作为真实行数的百分比.

Note: If Out-DataTable requires a nontrivial amount of processing time after the last object has been passed to it, you could pad the number of rows for the purpose of the percentage display based on an estimate of that additional time, as a percentage of the true row count.

这篇关于如何将当前项目返回到管道?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！