C ++高性能文件读写(C ++ 14) [英] C++ High Performance File Reading and Writing (C++14)

问题描述

我正在编写一个C ++ 14程序，用于从文件中加载文本字符串，对其进行一些计算，然后写回到另一个文件中.我使用的是Linux，文件相对较大(O(10 ^ 6行)).我典型的解决方法是使用旧的C getline和sscanf实用程序读取和解析输入，并使用fprintf(FILE*, …)写入输出文件.这行得通，但是我想知道是否有更好的方法可以实现高性能的目标，并且通常会推荐使用我正在使用的现代C ++标准的方法.我听说iostream相当慢；如果是这样，我想知道是否有更推荐的方法.

I’m writing a C++14 program to load text strings from a file, do some computation on them, and write back to another file. I’m using Linux, and the files are relatively large (O(10^6 lines)). My typical approach to this is to use the old C getline and sscanf utilities to read and parse the input, and fprintf(FILE*, …) to write the output files. This works, but I’m wondering if there’s a better way with the goals of high performance and generally recommended approach with the modern C++ standard that I’m using. I’ve heard that iostream is quite slow; if that’s true, I’m wondering if there’s a more recommended approach.

更新:为了澄清用例:对于输入文件的每一行，我将进行一些文本操作(数据清除等).每条线都是独立的.因此，加载整个输入文件(或至少大块文件)，并逐行处理它，然后编写它，似乎是最有意义的.理想的抽象方法是将迭代器添加到读缓冲区，每行都是一个条目.有没有推荐的方法可以使用std :: ifstream做到这一点?

Update: To clarify a bit on the use case: for each line of the input file, I'll be doing some text manipulation (data cleanup, etc.). Each line is independent. So, loading the entire input file (or, at least large chunks of it), and processing it line by line, and then writing it, seems to make the most sense. The ideal abstraction for this would be to get an iterator to the read-in buffer, with each line being an entry. Is there a recommended way to do that with std::ifstream?

推荐答案

如果您有足够的内存来执行此操作，最快的选择是将整个文件读入1次读取的缓冲区中，然后处理内存中的缓冲区，然后用1次写入再次将其全部写入.

The fastest option, if you have the memory to do it, is to read the entire file into a buffer with 1 read, process the buffer in memory, and write it all out again with 1 write.

全部阅读:

std::string buffer;

std::ifstream f("file.txt");
f.seekg(0, std::ios::end);
buffer.resize(f.tellg());
f.seekg(0);
f.read(buffer.data(), buffer.size());

然后处理

然后全部编写:

std::ofstream f("file.txt");
f.write(buffer.data(), buffer.size());

这篇关于C ++高性能文件读写(C ++ 14)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！