如何让IOStream执行更好？ [英] How to get IOStream to perform better?
问题描述
以前的C用户喜欢使用 printf / scanf 系列函数，甚至在C ++中。 p>
虽然我承认我发现接口的方式更好（特别是类似POSIX的格式和本地化），但似乎压倒性的关注是性能。
看看他的问题：
似乎最好的答案是使用 fscanf ... C ++ ifstream 始终是x2-x3倍慢。
我认为如果我们可以编译
反射的镜头是：
ul> rdbuf（） - > pubsetbuf（buffer，size）？） std :: ios_base :: sync_with_stdio ？）当然，欢迎其他方法。
注意：由Dietmar Kuhl提到的一个新实现，但我无法找到关于它的更多细节，以前的引用似乎是一个死链接
这是我到目前为止收集的：
缓冲 ：
如果默认情况下缓冲区非常小，增加缓冲区大小肯定可以提高性能：
- 它减少了硬盘命中次数
- 它减少了系统调用次数
可以通过访问底层的 streambuf 实现来设置缓冲区。
char Buffer [N];
std :: ifstream file（file.txt）;
file.rdbuf（） - > pubsetbuf（Buffer，N）;
// rdbuf的指针读取器保证
//成功构造函数后成为非null
b $ b
警告由@iavr提供：根据 cppreference ，最好在打开文件之前调用 pubsetbuf 。各种标准库实现的行为有所不同。
区域设置：
Locale可以执行字符转换，过滤，以及涉及数字或日期的更聪明的技巧。他们经历了一个复杂的动态调度和虚拟调用系统，因此删除它们可以帮助减少罚分。
默认 C locale意味着不执行任何转换以及跨机器统一。
同步：
我看不到
可以访问全局设置（ std :: ios_base 的静态成员）
code $> code 测量：
使用这个，我玩弄了一个简单的程序，使用 gcc 3.4.2 在SUSE 10p3上编译 -O2 。
C：7.76532e + 06
C ++：1.0874e + 07
这代表了减少大约 20％ ...用于默认代码。实际上，篡改缓冲区（在C或C ++中）或同步参数（C ++）没有产生任何改善。
其他人的结果：
@Irfy on g ++ 4.7.2-2ubuntu1，-O3，虚拟化Ubuntu 11.10,3.5.0-25通用，x86_64，足够ram / cpu，196MB几个find / >> largefile.txt运行
C：634572
C ++：473222
C ++ 速度快25％
> b $ b @ gatteo Italia on g ++ 4.4.5 ，-O3，Ubuntu Linux 10.10 x86_64和随机的180 MB文件
C：910390
C ++：776016
C ++ 速度快17％
@ Bogatyr on g ++ i686-apple-darwin10-g ++ - 4.2.1（GCC）4.2.1（Apple Inc. build 5664），mac mini，4GB ram，空闲，除了此测试带有168MB数据文件
C：4.34151e + 06
C ++：9.14476e + 06
< 111％较慢
@Asu on clang ++ 3.8.0-2ubuntu4，Kubuntu 16.04 Linux 4.8-rc3,8GB ram，i5 Haswell，Crucial SSD，88MB数据文件（tar.xz存档）
C：270895
C ++：162799
C ++ 更快速66％
因此答案是：问题，真的取决于平台：/
这里的代码对于那些有兴趣的基准测试：
#include< fstream>
#include< iostream>
#include< iomanip>
#include< cmath>
#include< cstdio>
#include< sys / time.h>
template< typename Func>
double benchmark（Func f，size_t iterations）
{
f（）;
timeval a，b;
gettimeofday（& a，0）;
for（; iterations - > 0;）
{
f（）;
}
gettimeofday（& b，0）;
return（b.tv_sec *（unsigned int）1e6 + b.tv_usec） -
（a.tv_sec *（unsigned int）1e6 + a.tv_usec）;
}
struct CRead
{
CRead（char const * filename）：_filename（filename）{}
void operator（）（）{
FILE * file = fopen（_filename，r）;
int count = 0;
while（fscanf（file，％s，_buffer）== 1）{++ count; }
fclose（file）;
}
char const * _filename;
char _buffer [1024];
};
struct CppRead
{
CppRead（char const * filename）：_filename（filename），_buffer（）{}
enum {BufferSize = 16184 };
void operator（）（）{
std :: ifstream file（_filename，std :: ifstream :: in）;
//注释删除扩展缓冲区
file.rdbuf（） - > pubsetbuf（_buffer，BufferSize）;
intcount = 0;
std :: string s;
while（file>> s）{++ count; }
}
char const * _filename;
char _buffer [BufferSize];
};
int main（int argc，char * argv []）
{
size_t iterations = 1;
if（argc> 1）{iterations = atoi（argv [1]）; }
char const * oldLocale = setlocale（LC_ALL，C）;
if（strcmp（oldLocale，C）！= 0）{
std :: cout< 替换旧地区< oldLocale< 'by'C'\\\
;
}
char const * filename =largefile.txt;
CRead cread（filename）;
CppRead cppread（filename）;
//注释使用默认设置
bool oldSyncSetting = std :: ios_base :: sync_with_stdio（false）;
double ctime = benchmark（cread，iterations）;
double cpptime = benchmark（cppread，iterations）;
//如果oldSyncSetting的声明被注释，则注释
std :: ios_base :: sync_with_stdio（oldSyncSetting）;
std :: cout<< C：< ctime< \\\
C ++：< cpptime<< \\\
;
return 0;
}
Most previously C-users prefer to use the printf / scanf family of functions even in C++.
Although I admit that I find the interface way better (especially POSIX-like format and localization), it seems that an overwhelming concern is performance.
Taking at look at his question:
It seems that the best answer is to use fscanf... and that the C++ ifstream is consistently x2-x3 times slower.
I thought it would be great if we could compile a repository of "tips" to improve IOStreams performance: what work, what does not.
Pists of reflexion are:
- buffering (
rdbuf()->pubsetbuf(buffer, size) ?)
- synchronization (
std::ios_base::sync_with_stdio ?)
- locale handling (Could we use a trimmed-down locale ? Remove it altogether ?)
And of course, other approaches are welcome.
Note: an "new" implementation, by Dietmar Kuhl, was mentioned, but I was unable to locate much details about it, previous references seem to be a dead link
解决方案 Here is what I have gathered so far:
Buffering:
If by default the buffer is very small, increasing the buffer size can definitely improve the performance:
- it reduces the number of HDD hits
- it reduces the number of system calls
Buffer can be set by accessing the underlying streambuf implementation.
char Buffer[N];
std::ifstream file("file.txt");
file.rdbuf()->pubsetbuf(Buffer, N);
// the pointer reader by rdbuf is guaranteed
// to be non-null after successful constructor
Warning courtesy of @iavr: according to cppreference it is best to call pubsetbuf before opening the file. Various standard library implementations otherwise have different behaviors.
Locale Handling:
Locale can perform character conversion, filtering, and more clever tricks where numbers or dates are involved. They go through a complex system of dynamic dispatch and virtual calls, so removing them can help trimming down the penalty hit.
The default C locale is meant not to perform any conversion as well as being uniform across machines. It's a good default to use.
Synchronization:
I could not see any performance improvement using this facility.
One can access a global setting (static member of std::ios_base) using the sync_with_stdio static function.
Measurements:
Playing with this, I have toyed with a simple program, compiled using gcc 3.4.2 on SUSE 10p3 with -O2.
C : 7.76532e+06
C++: 1.0874e+07
Which represents a slowdown of about 20%... for the default code. Indeed tampering with the buffer (in either C or C++) or the synchronization parameters (C++) did not yield any improvement.
Results by others:
@Irfy on g++ 4.7.2-2ubuntu1, -O3, virtualized Ubuntu 11.10, 3.5.0-25-generic, x86_64, enough ram/cpu, 196MB of several "find / >> largefile.txt" runs
C : 634572
C++: 473222
C++ 25% faster
@Matteo Italia on g++ 4.4.5, -O3, Ubuntu Linux 10.10 x86_64 with a random 180 MB file
C : 910390
C++: 776016
C++ 17% faster
@Bogatyr on g++ i686-apple-darwin10-g++-4.2.1 (GCC) 4.2.1 (Apple Inc. build 5664), mac mini, 4GB ram, idle except for this test with a 168MB datafile
C : 4.34151e+06
C++: 9.14476e+06
C++ 111% slower
@Asu on clang++ 3.8.0-2ubuntu4, Kubuntu 16.04 Linux 4.8-rc3, 8GB ram, i5 Haswell, Crucial SSD, 88MB datafile (tar.xz archive)
C : 270895
C++: 162799
C++ 66% faster
So the answer is: it's a quality of implementation issue, and really depends on the platform :/
The code in full here for those interested in benchmarking:
#include <fstream>
#include <iostream>
#include <iomanip>
#include <cmath>
#include <cstdio>
#include <sys/time.h>
template <typename Func>
double benchmark(Func f, size_t iterations)
{
f();
timeval a, b;
gettimeofday(&a, 0);
for (; iterations --> 0;)
{
f();
}
gettimeofday(&b, 0);
return (b.tv_sec * (unsigned int)1e6 + b.tv_usec) -
(a.tv_sec * (unsigned int)1e6 + a.tv_usec);
}
struct CRead
{
CRead(char const* filename): _filename(filename) {}
void operator()() {
FILE* file = fopen(_filename, "r");
int count = 0;
while ( fscanf(file,"%s", _buffer) == 1 ) { ++count; }
fclose(file);
}
char const* _filename;
char _buffer[1024];
};
struct CppRead
{
CppRead(char const* filename): _filename(filename), _buffer() {}
enum { BufferSize = 16184 };
void operator()() {
std::ifstream file(_filename, std::ifstream::in);
// comment to remove extended buffer
file.rdbuf()->pubsetbuf(_buffer, BufferSize);
int count = 0;
std::string s;
while ( file >> s ) { ++count; }
}
char const* _filename;
char _buffer[BufferSize];
};
int main(int argc, char* argv[])
{
size_t iterations = 1;
if (argc > 1) { iterations = atoi(argv[1]); }
char const* oldLocale = setlocale(LC_ALL,"C");
if (strcmp(oldLocale, "C") != 0) {
std::cout << "Replaced old locale '" << oldLocale << "' by 'C'\n";
}
char const* filename = "largefile.txt";
CRead cread(filename);
CppRead cppread(filename);
// comment to use the default setting
bool oldSyncSetting = std::ios_base::sync_with_stdio(false);
double ctime = benchmark(cread, iterations);
double cpptime = benchmark(cppread, iterations);
// comment if oldSyncSetting's declaration is commented
std::ios_base::sync_with_stdio(oldSyncSetting);
std::cout << "C : " << ctime << "\n"
"C++: " << cpptime << "\n";
return 0;
}
这篇关于如何让IOStream执行更好？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
