如何让 IOStream 性能更好? [英] How to get IOStream to perform better?
问题描述
大多数学习 C 的 C++ 用户更喜欢使用 printf/scanf 系列函数，即使他们在使用 C++ 编码.
虽然我承认我发现界面更好(尤其是类似 POSIX 的格式和本地化)，但似乎压倒性的关注是性能.
看看这个问题:
<块引用>似乎最好的答案是使用 fscanf 并且 C++ ifstream 始终慢 2-3 倍.
我认为如果我们可以编译一个技巧"存储库来提高 IOStreams 性能，哪些有效，哪些无效.
要考虑的要点
- 缓冲 (
rdbuf()->pubsetbuf(buffer, size)) - 同步(
std::ios_base::sync_with_stdio) - 语言环境处理(我们可以使用精简的语言环境，还是完全删除它?)
当然，欢迎使用其他方法.
注意:提到了 Dietmar Kuhl 的新"实现，但我无法找到有关它的许多细节.以前的参考文献似乎是死链接.
这是我目前收集到的:
缓冲:
如果默认缓冲区很小，增加缓冲区大小肯定可以提高性能:
- 它减少了硬盘命中次数
- 减少系统调用次数
可以通过访问底层的streambuf实现来设置缓冲区.
char Buffer[N];std::ifstream 文件("file.txt");file.rdbuf()->pubsetbuf(Buffer, N);//保证 rdbuf 的指针读取器//构造函数成功后非空@iavr 提供的警告:根据 cppreference最好在打开文件之前调用 pubsetbuf.否则，各种标准库实现具有不同的行为.
区域设置处理:
Locale 可以执行字符转换、过滤以及涉及数字或日期的更聪明的技巧.它们经历了一个复杂的动态调度和虚拟调用系统，因此删除它们有助于减少惩罚.
默认的 C 语言环境旨在不执行任何转换以及跨机器统一.这是一个很好的默认使用.
同步:
我看不到使用此工具的任何性能改进.
可以使用sync_with_stdio 静态函数访问全局 设置(std::ios_base 的静态成员).
测量:
玩这个，我玩弄了一个简单的程序，使用 gcc 3.4.2 在 SUSE 10p3 和 -O2 上编译.
C : 7.76532e+06
C++:1.0874e+07
对于默认代码，这代表了大约 20%... 的减速.实际上，篡改缓冲区(在 C 或 C++ 中)或同步参数 (C++) 并没有产生任何改进.
其他人的结果:
<块引用>@Irfy on g++ 4.7.2-2ubuntu1，-O3，虚拟化 Ubuntu 11.10，3.5.0-25-generic，x86_64，足够的 ram/cpu，196MB 的几个find/>> largefile.txt"运行
C:634572C++:473222
C++ 快 25%
<块引用>@Matteo Italia 在 g++ 4.4.5、-O3、Ubuntu Linux 10.10 x86_64 上使用一个随机的 180 MB 文件
电话:910390
C++:776016
C++ 快 17%
<块引用>@Bogatyr on g++ i686-apple-darwin10-g++-4.2.1 (GCC) 4.2.1(Apple Inc. build 5664)，mac mini，4GB 内存，空闲，除了这个测试有 168MB 数据文件
C : 4.34151e+06
C++:9.14476e+06
C++ 慢 111%
<块引用>@Asu on clang++ 3.8.0-2ubuntu4、Kubuntu 16.04 Linux 4.8-rc3、8GB ram、i5 Haswell、Crucial SSD、88MB 数据文件(tar.xz 存档)
C : 270895C++:162799
C++ 快 66%
所以答案是:这是一个实施质量问题，实际上取决于平台:/
对于那些对基准测试感兴趣的人，这里有完整的代码:
#include #include #include #include <cmath>#include #include 模板 双基准(Func f，size_t 迭代){F();时间值 a, b;gettimeofday(&a, 0);for (; 迭代 --> 0;){F();}gettimeofday(&b, 0);返回 (b.tv_sec * (unsigned int)1e6 + b.tv_usec) -(a.tv_sec * (unsigned int)1e6 + a.tv_usec);}结构 CRead{CRead(char const* 文件名): _filename(filename) {}无效运算符()(){FILE* file = fopen(_filename, "r");整数计数 = 0;while ( fscanf(file,"%s", _buffer) == 1 ) { ++count;}fclose(文件)；}字符常量* _文件名；字符_缓冲区[1024]；};结构 CppRead{CppRead(char const* filename): _filename(filename), _buffer() {}枚举 { BufferSize = 16184 };无效运算符()(){std::ifstream 文件(_filename, std::ifstream::in);//删除扩展缓冲区的注释file.rdbuf()->pubsetbuf(_buffer, BufferSize);整数计数 = 0;std::string s;而(文件>> s){ ++计数;}}字符常量* _文件名；char _buffer[BufferSize];};int main(int argc, char* argv[]){size_t 迭代次数 = 1;if (argc > 1) { 迭代 = atoi(argv[1]);}char const* oldLocale = setlocale(LC_ALL,"C");如果(strcmp(oldLocale，C")！= 0){std::cout <<替换了旧的语言环境"<<oldLocale<<"' by 'C'
";}char const* 文件名 = "大文件.txt";CRead 读取(文件名)；CppRead cppread(文件名);//注释以使用默认设置bool oldSyncSetting = std::ios_base::sync_with_stdio(false);双 ctime = 基准(创建，迭代)；双 cpptime = 基准(cppread，迭代)；//如果注释了 oldSyncSetting 的声明，则注释std::ios_base::sync_with_stdio(oldSyncSetting);std::cout <<C:"<<时间<<"
"C++:"<<cpptime<<"
";返回0；} Most C++ users that learned C prefer to use the printf / scanf family of functions even when they're coding in C++.
Although I admit that I find the interface way better (especially POSIX-like format and localization), it seems that an overwhelming concern is performance.
Taking at look at this question:
It seems that the best answer is to use fscanf and that the C++ ifstream is consistently 2-3 times slower.
I thought it would be great if we could compile a repository of "tips" to improve IOStreams performance, what works, what does not.
Points to consider
- buffering (
rdbuf()->pubsetbuf(buffer, size)) - synchronization (
std::ios_base::sync_with_stdio) - locale handling (Could we use a trimmed-down locale, or remove it altogether ?)
Of course, other approaches are welcome.
Note: a "new" implementation, by Dietmar Kuhl, was mentioned, but I was unable to locate many details about it. Previous references seem to be dead links.
Here is what I have gathered so far:
Buffering:
If by default the buffer is very small, increasing the buffer size can definitely improve the performance:
- it reduces the number of HDD hits
- it reduces the number of system calls
Buffer can be set by accessing the underlying streambuf implementation.
char Buffer[N];
std::ifstream file("file.txt");
file.rdbuf()->pubsetbuf(Buffer, N);
// the pointer reader by rdbuf is guaranteed
// to be non-null after successful constructor
Warning courtesy of @iavr: according to cppreference it is best to call pubsetbuf before opening the file. Various standard library implementations otherwise have different behaviors.
Locale Handling:
Locale can perform character conversion, filtering, and more clever tricks where numbers or dates are involved. They go through a complex system of dynamic dispatch and virtual calls, so removing them can help trimming down the penalty hit.
The default C locale is meant not to perform any conversion as well as being uniform across machines. It's a good default to use.
Synchronization:
I could not see any performance improvement using this facility.
One can access a global setting (static member of std::ios_base) using the sync_with_stdio static function.
Measurements:
Playing with this, I have toyed with a simple program, compiled using gcc 3.4.2 on SUSE 10p3 with -O2.
C : 7.76532e+06
C++: 1.0874e+07
Which represents a slowdown of about 20%... for the default code. Indeed tampering with the buffer (in either C or C++) or the synchronization parameters (C++) did not yield any improvement.
Results by others:
@Irfy on g++ 4.7.2-2ubuntu1, -O3, virtualized Ubuntu 11.10, 3.5.0-25-generic, x86_64, enough ram/cpu, 196MB of several "find / >> largefile.txt" runs
C : 634572 C++: 473222
C++ 25% faster
@Matteo Italia on g++ 4.4.5, -O3, Ubuntu Linux 10.10 x86_64 with a random 180 MB file
C : 910390
C++: 776016
C++ 17% faster
@Bogatyr on g++ i686-apple-darwin10-g++-4.2.1 (GCC) 4.2.1 (Apple Inc. build 5664), mac mini, 4GB ram, idle except for this test with a 168MB datafile
C : 4.34151e+06
C++: 9.14476e+06
C++ 111% slower
@Asu on clang++ 3.8.0-2ubuntu4, Kubuntu 16.04 Linux 4.8-rc3, 8GB ram, i5 Haswell, Crucial SSD, 88MB datafile (tar.xz archive)
C : 270895 C++: 162799
C++ 66% faster
So the answer is: it's a quality of implementation issue, and really depends on the platform :/
The code in full here for those interested in benchmarking:
#include <fstream>
#include <iostream>
#include <iomanip>
#include <cmath>
#include <cstdio>
#include <sys/time.h>
template <typename Func>
double benchmark(Func f, size_t iterations)
{
f();
timeval a, b;
gettimeofday(&a, 0);
for (; iterations --> 0;)
{
f();
}
gettimeofday(&b, 0);
return (b.tv_sec * (unsigned int)1e6 + b.tv_usec) -
(a.tv_sec * (unsigned int)1e6 + a.tv_usec);
}
struct CRead
{
CRead(char const* filename): _filename(filename) {}
void operator()() {
FILE* file = fopen(_filename, "r");
int count = 0;
while ( fscanf(file,"%s", _buffer) == 1 ) { ++count; }
fclose(file);
}
char const* _filename;
char _buffer[1024];
};
struct CppRead
{
CppRead(char const* filename): _filename(filename), _buffer() {}
enum { BufferSize = 16184 };
void operator()() {
std::ifstream file(_filename, std::ifstream::in);
// comment to remove extended buffer
file.rdbuf()->pubsetbuf(_buffer, BufferSize);
int count = 0;
std::string s;
while ( file >> s ) { ++count; }
}
char const* _filename;
char _buffer[BufferSize];
};
int main(int argc, char* argv[])
{
size_t iterations = 1;
if (argc > 1) { iterations = atoi(argv[1]); }
char const* oldLocale = setlocale(LC_ALL,"C");
if (strcmp(oldLocale, "C") != 0) {
std::cout << "Replaced old locale '" << oldLocale << "' by 'C'
";
}
char const* filename = "largefile.txt";
CRead cread(filename);
CppRead cppread(filename);
// comment to use the default setting
bool oldSyncSetting = std::ios_base::sync_with_stdio(false);
double ctime = benchmark(cread, iterations);
double cpptime = benchmark(cppread, iterations);
// comment if oldSyncSetting's declaration is commented
std::ios_base::sync_with_stdio(oldSyncSetting);
std::cout << "C : " << ctime << "
"
"C++: " << cpptime << "
";
return 0;
}
这篇关于如何让 IOStream 性能更好?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
