如何在PowerShell中将一个大的文本文件分割成多个文件 [英] How to split one big text file into multiple files in powershell

问题描述

你好我有一个像这样的大文本文件

Hello I have one big text file like this

BIGFILE.TXT

COLUMN1,COLUMN2,COLUMN3,COLUMN4,COLUMN5,COLUMN6,COLUMN7,COLUMN8
11/24/2013,50.67,51.22,50.67,51.12,17,0,FILE1
11/25/2013,51.34,51.91,51.09,51.87,23,0,FILE1
12/30/2013,51.76,51.82,50.86,51.15,13,0,FILE1
12/31/2013,51.15,51.33,50.45,50.76,18,0,FILE1
1/1/2014,50.92,51.58,50.84,51.1,19,0,FILE2
1/4/2014,51.39,51.46,50.95,51.21,14,0,FILE2
1/7/2014,51.08,51.2,49.84,50.05,35,0,FILE2
1/8/2014,50.14,50.94,50.01,50.78,100,0,FILE3
1/11/2014,50.63,51.41,50.52,51.3,190,0,FILE3
1/15/2014,54.03,55.74,53.69,54.93,110,0,FILE4
1/19/2014,53.67,54.19,53.55,53.82,24,0,FILE4
1/20/2014,53.83,54.26,53.47,53.53,23,0,FILE4
1/21/2014,53.8,54.55,53.7,54.1,24,0,FILE4
1/26/2014,53.26,53.93,53.23,53.65,31,0,FILE5
1/27/2014,53.78,54,53.64,53.81,110,0,FILE5

我正在寻找如何将这个文件分割成多个tex t文件。在这种情况下，一个文件将被分成5个文本文件。每个文本文件的名称将从第8列获取。大文件是逗号分隔的。所以输出结果是：

I'm looking for way how to split this file into multiple text files. In this case one file would be split into 5 text files. Name of each text file would be taken from column number 8. The big file is comma delimited. So the output would be:

FILE1.txt

COLUMN1,COLUMN2,COLUMN3,COLUMN4,COLUMN5,COLUMN6,COLUMN7,COLUMN8
11/24/2013,50.67,51.22,50.67,51.12,17,0,FILE1
11/25/2013,51.34,51.91,51.09,51.87,23,0,FILE1
12/30/2013,51.76,51.82,50.86,51.15,13,0,FILE1
12/31/2013,51.15,51.33,50.45,50.76,18,0,FILE1

FILE2.TXT

COLUMN1,COLUMN2,COLUMN3,COLUMN4,COLUMN5,COLUMN6,COLUMN7,COLUMN8
1/1/2014,50.92,51.58,50.84,51.1,19,0,FILE2
1/4/2014,51.39,51.46,50.95,51.21,14,0,FILE2
1/7/2014,51.08,51.2,49.84,50.05,35,0,FILE2

FILE3.TXT

COLUMN1,COLUMN2,COLUMN3,COLUMN4,COLUMN5,COLUMN6,COLUMN7,COLUMN8
1/8/2014,50.14,50.94,50.01,50.78,100,0,FILE3
1/11/2014,50.63,51.41,50.52,51.3,190,0,FILE3
.
.
.

大文本文件有几千行。
有没有人有一些想法如何做到这一点？

The big text file has several thousand of lines. Does anybody have some idea how to do that?

感谢您的帮助。
J。

Thank you for help. J.

推荐答案

这正是我在Bob McCoy帮助下寻找的东西。 b
$ b

Here is exactly what I was looking for based on help from Bob McCoy

#  Split-File.ps1

$src = "C:\Ephemeral\bigfile.csv"
$dstDir = "C:\Ephemeral\files\"

# Delete previous output files
Remove-Item -Path "$dstDir\\*"

# Read input and create subordinate files based on column 8 content
$header = Get-Content -Path $src | select -First 1

Get-Content -Path $src | select -Skip 1 | foreach {
    $file = "$(($_ -split ",")[7]).txt"
    Write-Verbose "Wrting to $file"
    if (-not (Test-Path -Path $dstDir\$file))
    {
        Out-File -FilePath $dstDir\$file -InputObject $header -Encoding ascii
    }
    Out-File -FilePath $dstDir\$file -InputObject $_ -Encoding ascii -Append
}

这个代码只是小问题。花了差不多80分钟把我的大文件分成1800个小文件，所以如果有人有一些建议如何提高这个代码的性能，将不胜感激。 Mayby这将有助于bigfile按第8列按字母顺序排序。所有小文件的名字也存储在＃8列。

There is just small issue with this code. It took almost 80 minutes to split my big file into 1800 small files so if anybody has some suggestion how to increase performance of this code it would be highly appreciated. Mayby it would help that "bigfile" is sorted alphabetically by column#8. And names for all small files are also stored in column#8.

这篇关于如何在PowerShell中将一个大的文本文件分割成多个文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！