读取大型txt文件多线程？ [英] Read large txt file multithreaded?

问题描述

我有100000行的大型txt文件。
我需要启动n个线程并为此文件中的每个线程分配唯一的行。

I have large txt file with 100000 lines. I need to start n-count of threads and give every thread unique line from this file.

执行此操作的最佳方法是什么？我想我需要逐行读取文件，并且迭代器必须是全局的才能锁定它。将文本文件加载到列表将非常耗时，并且我会收到 OutofMemory 异常。有任何想法吗？

What is the best way to do this? I think I need to read file line by line and iterator must be global to lock it. Loading the text file to list will be time-consuming and I can receive OutofMemory exception. Any ideas?

推荐答案

您可以使用 File.ReadLines方法逐行读取文件，而无需立即将整个文件加载到内存中，并且 Parallel.ForEach方法来并行处理多个线程中的行：

You can use the File.ReadLines Method to read the file line-by-line without loading the whole file into memory at once, and the Parallel.ForEach Method to process the lines in multiple threads in parallel:

Parallel.ForEach(File.ReadLines("file.txt"), (line, _, lineNumber) =>
{
    // your code here
});

这篇关于读取大型txt文件多线程？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！