读取输入文件，最快的方法? [英] read input files, fastest way possible?

问题描述

我有许多浮点数形式的数据文本文件.我正在寻找用 C++ 读取它们的最快方法.如果这是最快的，我可以将文件更改为二进制文件.

I have numerous text files of data in the form of float numbers. I'm looking for the fastest way to read them in C++. I can change the file to binary if that's the fastest.

如果您能给我提示或将我推荐给一个有完整解释的网站，那就太好了.我不知道是否有任何库可以快速完成工作.即使有任何开源软件可以完成这项工作，也会有所帮助.

It would be great if you could give me hint or refer me to a website with complete explanation. I don't know whether there is any library that does the work fast. Even if there is any open source software that does the work, that would be helpful.

推荐答案

拥有二进制文件是最快的选择.您不仅可以在单个操作中使用原始 istream::read 直接在数组中读取它(非常快)，而且如果您的操作系统支持，您甚至可以将文件映射到内存中；您可以在 POSIX 系统上使用 open/mmap，CreateFile/CreateFileMapping/MapViewOfFile在 Windows 上，甚至是 Boost 跨平台解决方案(感谢 @Cory Nelson 指出).

Having a binary file is the fastest option. Not only you can read it directly in an array with a raw istream::read in a single operation (which is very fast), but you can even map the file in memory if your OS supports it; you can use open/mmap on POSIX systems, CreateFile/CreateFileMapping/MapViewOfFile on Windows, or even the Boost cross-platform solution (thanks @Cory Nelson for pointing it out).

快速&脏示例，假设文件包含一些 float 的原始表示:

Quick & dirty examples, assuming the file contains the raw representation of some floats:

正常"阅读:

#include <fstream>
#include <vector>

// ...

// Open the stream
std::ifstream is("input.dat");
// Determine the file length
is.seekg(0, std::ios_base::end);
std::size_t size=is.tellg();
is.seekg(0, std::ios_base::beg);
// Create a vector to store the data
std::vector<float> v(size/sizeof(float));
// Load the data
is.read((char*) &v[0], size);
// Close the file
is.close();

使用共享内存:

#include <boost/interprocess/file_mapping.hpp>
#include <boost/interprocess/mapped_region.hpp>

using boost::interprocess;

// ....

// Create the file mapping
file_mapping fm("input.dat", read_only);
// Map the file in memory
mapped_region region(fm, read_only);
// Get the address where the file has been mapped
float * addr = (float *)region.get_address();
std::size_t elements  = region.get_size()/sizeof(float);

这篇关于读取输入文件，最快的方法?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！