读取大 TXT 文件，内存不足异常 [英] Read Big TXT File, Out of Memory Exception

问题描述

我想读取 500 MB 的大 TXT 文件，首先我使用

I want to read big TXT file size is 500 MB, First I use

var file = new StreamReader(_filePath).ReadToEnd();  
var lines = file.Split(new[] { '
' });

但它抛出内存异常然后我尝试逐行读取但在读取大约 150 万行后再次抛出内存异常

but it throw out of memory Exception then I tried to read line by line but again after reading around 1.5 million lines it throw out of memory Exception

  using (StreamReader r = new StreamReader(_filePath))
         {            
             while ((line = r.ReadLine()) != null)            
                 _lines.Add(line);            
         }

或者我用过

  foreach (var l in File.ReadLines(_filePath))
            {
                _lines.Add(l);
            }

但是我又收到了

System.OutOfMemoryException"类型的异常发生在mscorlib.dll 但未在用户代码中处理

An exception of type 'System.OutOfMemoryException' occurred in mscorlib.dll but was not handled in user code

我的机器是强大的机器，有 8GB 内存，所以这不应该是我的机器问题.

My Machine is powerful machine with 8GB of ram so it shouldn't be my machine problem.

ps:我尝试在 NotePadd++ 中打开此文件，但收到文件太大而无法打开"异常.

p.s: I tried to open this file in NotePadd++ and I received 'the file is too big to be opened' exception.

推荐答案

只需使用 File.ReadLines 返回一个 IEnumerable 并且不会一次将所有行加载到内存中.

Just use File.ReadLines which returns an IEnumerable<string> and doesn't load all the lines at once to the memory.

foreach (var line in File.ReadLines(_filePath))
{
    //Don't put "line" into a list or collection.
    //Just make your processing on it.
}

这篇关于读取大 TXT 文件，内存不足异常的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！