读取浮点数更快的文本文件，并存储在浮点数 [英] Reading text file with floating point numbers faster and storing in vector of floats

问题描述

我有一个在visual studio 2010中编写的c ++代码，它读取一个文本文件（包含由空格分隔的数万个浮点数）.Code读取文本文件内容并将其存储到浮点数的向量中。我的问题是，代码需要花费大量的时间来读取和复制到向量。有一个更快的方法来做这个。可以在视觉工作室c + +（使用boost库或mmap）完成

I have a c++ code written in visual studio 2010, which reads a text file ( which contains tens of thousands of floating point numbers separated by space).Code reads the text file contents and store it to a vector of floating points.My problem is , code is taking alot of time to read and copy to the vector.Is there a faster way to do this.Some thing that can be done in visual studio c++ ( using boost libraries or mmap )

vector<float> ReplayBuffer;
ifstream in;
in.open("fileName.txt");
if(in.is_open())
{
    in.setf(ios::fixed);
    in.precision(3);

   in.seekg(0,ios::end);
   fileSizes = in.tellg();

   in.seekg(0,ios::beg);
   while(!in.eof())
   {
   for(float f;in>>f;)
       ReplayBuffer.push_back(f);
   }
   in.close();
}

推荐答案

地址空间，您可以在生成的内存上 mmap it和
然后使用 istrstream 。 istrstream 是
正式弃用，但它仍然存在，并且是唯一的
标准流，将在这里工作。或者你可以编写自己的
内存streambuf，这可能比 istrstream ，
更快，因为你不必支持查找等。 （虽然
在上寻找istrstream 也是一个相当微不足道的操作，
，并且不应该影响其余的）。

If the file fits in your address space, you can mmap it and then use istrstream on the resulting memory. istrstream is formally deprecated, but it's still there, and is the only standard stream that will work here. Or you can write your own memory streambuf, which might even be faster than istrstream, because you won't have to support seeking, etc. on it (although seeking on an istrstream is also a fairly trivial operation, and shouldn't impact on the rest very much).

除此之外，每一层的抽象通常花费
，所以它可能会更快（虽然不是
必然非常这样）if使用 strtod 手动循环。

Beyond that, every layer of abstraction generally costs something, so it will probably be even faster (although not necessarily very much so) if you loop manually, using strtod.

在所有情况下，将通用浮点转换为机器
浮点是一种昂贵的操作。如果你知道
关于你将看到的值和它们的格式（例如没有
科学记数法，值在一定范围内，最大
小数后的位数）可以编写比code> strtod 更快的
a转换例程。这个
需要一些注意，但是如果你知道数字中的
个小数位的总数将总是产生一个值，
将适合 int ，你可以做一个非常快速的int转换，
忽略'。'，然后乘以
适当的浮点值（例如，在'。'之后有
3位数字时，'.001'）。

In all cases, converting a generic floating point into machine floating point is an expensive operation. If you know something about the values you will be seeing, and their format (e.g. no scientific notation, values in a certain range, with a maximum number of places after the decimal), it's possible to write a conversion routine that would be faster than strtod. This requires some care, but if you know that the total number of decimal digits in the number will always result in a value that will fit in an int, you can do a very rapid int conversion, ignoring the '.', and then scale it by multiplying by the appropriate floating point value (e.g. '.001' if there were 3 digits after the '.').

这篇关于读取浮点数更快的文本文件，并存储在浮点数的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！