在Powershell中使用Import-CSV的更有效方法 [英] More efficient way of using Import-CSV in powershell

问题描述

我正在尝试找出一种使用Import-CSV(powershell)将值放入csv文件数组的更有效的方法.问题是这些文件中的一些文件有数十万行，并且与其他代码行一起运行此脚本似乎是一个很大的瓶颈.你们对如何使此代码更高效，更快地有何建议?

I am trying to figure out a more efficient way of using Import-CSV (powershell) to place values into an array of csv files. The problem is that some of these files have several hundred thousand lines and running this script in conjunction with other lines of code is what appears to be a big bottle neck. Do you guys have any suggestions of how to make this code more efficient and faster?

foreach($csv in $csvfiles)
{
    $csvname = $csv.name;
    $paygroup = $csvname.substring(4,3);
    $batch = $csvname.substring(14,4);
    write-host "Writing $csvname";
    $csvimportdata = Import-CSV $CurrentPath"\$csvname";
        foreach($record in $csvimportdata)
        {
        $record.chartfield1 = $paygroup;
        $record.chartfield2 = $batch;
        $record.chartfield3 = $record.line_descr.substring(0,6);
        }
    $csvimportdata | Export-CSV $CurrentPath"\$csvname" -NoTypeInformation
};

推荐答案

如果您的CSV很大，那么将其加载到内存中可能不是一个好主意.这样的事情怎么样:

If your CSVs are large then loading into memory is probably not a good idea. How about something like this:

foreach($csv in $csvfiles)
{
    $csvname = $csv.name
    $paygroup = $csvname.substring(4,3)
    $batch = $csvname.substring(14,4)
    write-host "Writing $csvname"

    Get-Content $CurrentPath"\$csvname" -Readcount 1 | % {
      # Regex below assumes a three column CSV
      $_ -replace '^([^,]+,[^,]+,[^,]{6}).*$', '$1'
    } | Set-Content $CurrentPath"\$csvname"
}

这篇关于在Powershell中使用Import-CSV的更有效方法的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！