Powershell Count行非常大的文件 [英] Powershell Count lines extremely large file
问题描述
我有一个大小为 250 GB 的超大文本文件,是供应商提供给我们的.他们还为我们提供了一个控制文件,该文件应该包含大文件中的行数.有时会出现不匹配.如何计算 Powershell 中的行数?我试过这个命令,运行了半个多小时,还没完成.
I have a extremely large text file of size 250 GB that's given to us by a vendor. They also give us a control file that is supposed to have the number of lines in the large file. Sometimes there is a mismatch. How do I count lines in Powershell? I tried this command and it ran for more than half hour and was not done yet.
Get-content C:\test.txt | Measure-Object –Line
(gc C:\test.txt | Measure-object | select count).count
感谢任何帮助谢谢先生
推荐答案
如果性能很重要,请避免使用 cmdlet 和管道;使用 switch -File
:
If performance matters, avoid the use of cmdlets and the pipeline; use switch -File
:
$count = 0
switch -File C:\test.txt { default { ++$count } }
switch -File
枚举指定文件的行;条件 default
匹配任何行.
switch -File
enumerates the lines of the specified file; condition default
matches any line.
为了感受性能差异:
# Create a sample file with 100,000 lines.
1..1e5 > tmp.txt
# Warm up the file cache
foreach ($line in [IO.File]::ReadLines("$pwd/tmp.txt")) { }
(Measure-Command { (Get-Content tmp.txt | Measure-Object).Count }).TotalSeconds
(Measure-Command { $count = 0; switch -File tmp.txt { default { ++$count } } }).TotalSeconds
来自我的 Windows 10/PSv5.1 机器的示例结果:
Sample results from my Windows 10 / PSv5.1 machine:
1.3081307 # Get-Content + Measure-Object
0.1097513 # switch -File
也就是说,在我的机器上,switch -File
命令快了大约 12 倍.
That is, on my machine the switch -File
command was about 12 times faster.
这篇关于Powershell Count行非常大的文件的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!