在windows中fwrite比WriteFile快吗？ [英] Is fwrite faster than WriteFile in windows?
问题描述
我一直认为WriteFile比fwrite更有效率，因为fwrite在内部调用WriteFile，但是下面的测试代码告诉我，fwrite比WriteFile更快。
fwrite成本为2毫秒，而WriteFile需要27000（FILE_ATTRIBUTE_NORMAL），每次写入调用后都会刷新。如果我调用WriteFile与FILE_FLAG_WRITE_THROUGH，并注释FlushFileBuffers（wfile）行，WriteFile会更快，它花费800.
所以是真的，fwrite调用WriteFile ？什么造成这么大的差别？ fwrite如何在内部工作？如何用比fwrite？（unbufferd，synchronous）更有效地用API将数据写入文件。
#include< h。
#include< stdio.h>
#include< iostream>
int main（）{
FILE * cfile = fopen（file1.txt，w）;
HANDLE wfile = CreateFile（file2.txt，GENERIC_WRITE，FILE_SHARE_READ，NULL，CREATE_ALWAYS，
/ * FILE_ATTRIBUTE_NORMAL * / FILE_FLAG_WRITE_THROUGH，NULL）;
DWORD written = 0;
DWORD start_time，end_time;
char * text =测试讯息哈哈哈哈;
int size = strlen（text）;
int times = 999;
start_Time = timeGetTime（）;
for（int i = 0; i
fflush（cfile）;
}
end_time = timeGetTime（）;
std :: cout<< end_time - start_time< '\\\
';
start_time = timeGetTime（）;
for（int i = 0; i
// FlushFileBuffers（wfile）;
}
end_time = timeGetTime（）;
std :: cout<< end_time - start_time< std :: endl;
系统（pause）;
return 0;
}
更新：答案，这里是答案：
参见VS目录\VS\crt\src\fflush.c：
//fflush.c
int __cdecl _fflush_nolock（FILE * str）{
//不相关的代码
if（str-> _flag& _IOCOMMIT）{
return （_commit（_fileno（str））？EOF：0）;
}
return 0;
}
因此这里是一个_IOCOMMIT标志，然后参见... \src\\ \\ fdopen.c
FILE * __cdecl _tfdopen（int filedes，const _TSCHAR * mode）{
//不相关代码
while（* ++ mode&& whileflag）
switch（* mode）{
// ...
case _T（'c'）：
if（cnflag）
whileflag = 0;
else {
cnflag = 1;
fileflag | = _IOCOMMIT;
}
break;
// ...
}
_tfopen由fopen内部调用，请参阅fopen的文档，我发现：
模式：
'c'
启用相关文件名的提交标志，以便在调用fflush或_flushall时将文件缓冲区的内容直接写入磁盘。
因此，只有当调用fopen时设置了'c'标志，才会调用_commit。
_commit函数最终调用FlushFileBuffers。
$ b $除了这些，我发现，当我只写几个数据到文件（不超过缓冲区大小），如果fwrite没有fflush，文本不会被明显地wrritten，而对于API，后WriteFile即使我不调用FlushFileBuffers，当我打开文件（程序在休眠），内容被自动写入文件，这是为什么我被困惑关于flush的原因之一，这个操作可以由操作系统完成， WriteFile将数据复制到系统缓存，并且其文件缓冲区由OS管理，因此fflush（）只是在没有实际刷新的情况下内部调用WriteFile，系统知道何时清除它们，也许当文件句柄关闭或另一个I / O访问此文件。所以我修改了基准如下：
start_time = timeGetTime
for（int i = 0; i
fflush（cfile）;
}
end_time = timeGetTime（）;
std :: cout<< end_time - start_time< '\\\
';
start_time = timeGetTime（）;
for（int i = 0; i
}
end_time = timeGetTime（）;
std :: cout<< end_time - start_time< std :: endl;
结果是
次：99999
fwrite：217
WriteFile：171
因此，总之，加快API文件写入操作：
-
不要显式调用FlushFileBuffers，需要时将系统缓存中的数据刷新到磁盘。
-
为WriteFile获取缓冲区，因为API调用花费的时间比简单的memcpy更多，当缓冲区填满时调用WriteFile。
使用Sysinternals中的进程监视器（procmon）等工具，你会看到对 fflush（）的调用不是与 FlushFileBuffers（wfile）或 FILE_FLAG_WRITE_THROUGH 标志 CreateFile（））。
fwrite（）将数据写入缓冲区，直到缓冲区填满，这将导致缓冲区中的数据发送到 WriteFile 调用。当调用 fflush（）时，发生的一切都是缓冲区中的数据传递给 WriteFile（） - fflush（）不调用 FlushFileBuffers（）：
1：21：32.9391534 AM test.exe 6132 WriteFile C：\temp\file1.txt SUCCESS Offset：0，Length：24
1:21 ：32.9392200 AM test.exe 6132 WriteFile C：\temp\file1.txt SUCCESS Offset：24，长度：24
1：21：32.9392340 AM test.exe 6132 WriteFile C：\temp\file1。 txt SUCCESS偏移：48，长度：24
1：21：32.9392436 AM test.exe 6132 WriteFile C：\temp\file1.txt SUCCESS偏移：72，长度：24
1:21： 32.9392526 AM test.exe 6132 WriteFile C：\temp\file1.txt SUCCESS Offset：96，长度：24
1：21：32.9392623 AM test.exe 6132 WriteFile C：\temp\file1.txt SUCCESS Offset：120，Length：24
为了进行比较， c $ c> fwrite（）循环，不带 fflush（）调用：
1：27：28.5675034 AM test.exe 3140 WriteFile C：\temp\file1.txt SUCCESS Offset：0，Length：1,024pre>
1：27：28.5676098 AM test。 exe 3140 WriteFile C：\temp\file1.txt SUCCESS Offset：1,024，Length：1,024
1：27：28.5676399 AM test.exe 3140 WriteFile C：\temp\file1.txt SUCCESS Offset：2,048 ，长度：1,024
1：27：28.5676651 AM test.exe 3140 WriteFile C：\temp\file1.txt SUCCESS偏移：3,072，长度：1,024
这里有一个来自
WriteFile（）循环（FILE_ATTRIBUTE_NORMAL 标志和显式调用FlushFileBuffers（）- 它只是使事情更容易看到，因为FlushFileBuffers（）调用显示在跟踪中，而不是显示为第二个4KBWriteFile（）调用）。1：21：29.0068503 AM test.exe 6132 WriteFile C：\temp\file2.txt SUCCESS Offset：0，Length：24，Priority：Normal
1：21：29.0069197 AM test.exe 6132 FlushBuffersFile C：\temp\file2.txt SUCCESS
1：21：29.0069517 AM test.exe 6132 WriteFile C：\temp\file2。 txt SUCCESS偏移：0，长度：4,096，I / O标志：非缓存，分页I / O，同步分页I / O，优先级：正常
1：21：29.0087574 AM test.exe 6132 WriteFile C： \temp\file2.txt SUCCESS Offset：24，长度：24
1：21：29.0087798 AM test.exe 6132 FlushBuffersFile C：\temp\file2.txt SUCCESS
1:21： 29.0088087 AM test.exe 6132 WriteFile C：\temp\file2.txt SUCCESS Offset：0，长度：4,096，I / O标志：非缓存，分页I / O，同步分页I / O，优先级：正常
1：21：29.0102260 AM test.exe 6132 WriteFile C：\temp\file2.txt SUCCESS偏移：48，长度：24
1：21：29.0102428 AM test.exe 6132 FlushBuffersFile C： \\temp\file2.txt SUCCESS
1：21：29.0102701 AM test.exe 6132 WriteFile C：\temp\file2.txt SUCCESS Offset：0，Length：4,096，I / O Flags：Non-cached ，分页I / O，同步分页I / O，优先级：正常
1：21：29.0113444 AM test.exe 6132 WriteFile C：\temp\file2.txt SUCCESS偏移量：72，长度：24
1：21：29.0113602 AM test.exe 6132 FlushBuffersFile C：\temp\file2.txt SUCCESS
1：21：29.0113848 AM test.exe 6132 WriteFile C：\temp\file2.txt成功偏移：0，长度：4,096，I / O标志：非缓存，分页I / O，同步分页I / O，优先级：正常
因此，你的基准显示了
WriteFile（）循环的严重缺点的原因是因为你有大约一千个调用Fush（）循环中的 $ f $（$I have always thought that WriteFile is more efficient than fwrite, because fwrite calls down to WriteFile internally, but the following test code show me that fwrite is faster significantly than WriteFile.
fwrite costs 2 milliseconds while WriteFile need 27000(FILE_ATTRIBUTE_NORMAL), both flushes after every write call. If I call WriteFile with FILE_FLAG_WRITE_THROUGH, and comment the FlushFileBuffers(wfile) line , WriteFile will be faster, it costs 800.
So Is it really that fwrite calls down to WriteFile? What makes such a huge difference? How does fwrite work internally? How can I write data to file with API more efficiently than fwrite?(unbufferd, synchronous).
#include <Windows.h> #include <stdio.h> #include <iostream> int main() { FILE* cfile = fopen("file1.txt", "w"); HANDLE wfile = CreateFile("file2.txt", GENERIC_WRITE, FILE_SHARE_READ, NULL, CREATE_ALWAYS, /*FILE_ATTRIBUTE_NORMAL*/FILE_FLAG_WRITE_THROUGH, NULL); DWORD written = 0; DWORD start_time, end_time; char * text = "test message ha ha ha ha"; int size = strlen(text); int times = 999; start_time = timeGetTime(); for(int i = 0; i < times; ++i) { fwrite(text, 1, size, cfile); fflush(cfile); } end_time = timeGetTime(); std::cout << end_time - start_time << '\n'; start_time = timeGetTime(); for(int i = 0; i < times; ++i) { WriteFile(wfile, text, size, &written, NULL); //FlushFileBuffers(wfile); } end_time = timeGetTime(); std::cout << end_time - start_time << std::endl; system("pause"); return 0; }Update: Thanks for the answers, here is the answer: see VS directory\VS\crt\src\fflush.c:
//fflush.c int __cdecl _fflush_nolock (FILE *str) { //irrelevant codes if (str->_flag & _IOCOMMIT) { return (_commit(_fileno(str)) ? EOF : 0); } return 0; }so here is a _IOCOMMIT flag, then see ...\src\fdopen.c
FILE * __cdecl _tfdopen (int filedes, const _TSCHAR *mode) { //irrelevant codes while(*++mode && whileflag) switch(*mode) { //... case _T('c'): if (cnflag) whileflag = 0; else { cnflag = 1; fileflag |= _IOCOMMIT; } break; //... }_tfopen is called by fopen internally, refer to the documents of fopen, I find this:
" mode: 'c'
Enable the commit flag for the associated filename so that the contents of the file buffer are written directly to disk if either fflush or _flushall is called." So, _commit is called only if 'c' flag is set when calling fopen.
the _commit function eventually calls FlushFileBuffers.
Besides these, I find that when I write only a few data to file(not exceeds the buffer size), if fwrite without fflush, the text will not be wrritten apparently, while for API, after WriteFile even if I don't call FlushFileBuffers, when I open the file(program is in Sleep), the content is written to file automatically, that was one of the reasons why I was confused about flush , this operation may be done by OS, WriteFile copies data to system cache, and its file buffer is managed by OS, so it's resonable that fflush() only calls WriteFile internally without real flushing, system knows when to flush them, maybe when file handle is closed or when another I/O access to this file is occured. So I modified the benchmark as this:
start_time = timeGetTime(); for(int i = 0; i < times; ++i) { fwrite(text, 1, size, cfile); fflush(cfile); } end_time = timeGetTime(); std::cout << end_time - start_time << '\n'; start_time = timeGetTime(); for(int i = 0; i < times; ++i) { WriteFile(wfile, text, size, &written, NULL); } end_time = timeGetTime(); std::cout << end_time - start_time << std::endl;the result is times:99999 fwrite:217 WriteFile:171
So, in conclusion, to speed up API file writting operation:
Do not call FlushFileBuffers explicitly， data in system cache will be flushed to disk when needed.
Get a buffer for WriteFile, just as fwrite does, because API call cost more time than simply memcpy, call WriteFile when buffer is filled up.
解决方案Using a tool like Process Monitor (procmon) from Sysinternals, you'll see that the call to
fflush()isn't doing the same thing asFlushFileBuffers(wfile)(or theFILE_FLAG_WRITE_THROUGHflag toCreateFile()).
fwrite()will write the data to a buffer until that buffer fills, which will cause it to send the data in the buffer toWriteFile()call. When you callfflush()all that happens is that the data currently in the buffer is passed to a call toWriteFile()-fflush()doesn't callFlushFileBuffers():1:21:32.9391534 AM test.exe 6132 WriteFile C:\temp\file1.txt SUCCESS Offset: 0, Length: 24 1:21:32.9392200 AM test.exe 6132 WriteFile C:\temp\file1.txt SUCCESS Offset: 24, Length: 24 1:21:32.9392340 AM test.exe 6132 WriteFile C:\temp\file1.txt SUCCESS Offset: 48, Length: 24 1:21:32.9392436 AM test.exe 6132 WriteFile C:\temp\file1.txt SUCCESS Offset: 72, Length: 24 1:21:32.9392526 AM test.exe 6132 WriteFile C:\temp\file1.txt SUCCESS Offset: 96, Length: 24 1:21:32.9392623 AM test.exe 6132 WriteFile C:\temp\file1.txt SUCCESS Offset: 120, Length: 24For comparison, here's an example of a trace from the
fwrite()loop without thefflush()call:1:27:28.5675034 AM test.exe 3140 WriteFile C:\temp\file1.txt SUCCESS Offset: 0, Length: 1,024 1:27:28.5676098 AM test.exe 3140 WriteFile C:\temp\file1.txt SUCCESS Offset: 1,024, Length: 1,024 1:27:28.5676399 AM test.exe 3140 WriteFile C:\temp\file1.txt SUCCESS Offset: 2,048, Length: 1,024 1:27:28.5676651 AM test.exe 3140 WriteFile C:\temp\file1.txt SUCCESS Offset: 3,072, Length: 1,024And here's a snippet of the trace from the
WriteFile()loop (with theFILE_ATTRIBUTE_NORMALflag and explicit call toFlushFileBuffers()- it just makes what's happening easier to see in the trace since theFlushFileBuffers()call is shown in the trace instead of just showing as a second 4KBWriteFile()call).1:21:29.0068503 AM test.exe 6132 WriteFile C:\temp\file2.txt SUCCESS Offset: 0, Length: 24, Priority: Normal 1:21:29.0069197 AM test.exe 6132 FlushBuffersFile C:\temp\file2.txt SUCCESS 1:21:29.0069517 AM test.exe 6132 WriteFile C:\temp\file2.txt SUCCESS Offset: 0, Length: 4,096, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Priority: Normal 1:21:29.0087574 AM test.exe 6132 WriteFile C:\temp\file2.txt SUCCESS Offset: 24, Length: 24 1:21:29.0087798 AM test.exe 6132 FlushBuffersFile C:\temp\file2.txt SUCCESS 1:21:29.0088087 AM test.exe 6132 WriteFile C:\temp\file2.txt SUCCESS Offset: 0, Length: 4,096, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Priority: Normal 1:21:29.0102260 AM test.exe 6132 WriteFile C:\temp\file2.txt SUCCESS Offset: 48, Length: 24 1:21:29.0102428 AM test.exe 6132 FlushBuffersFile C:\temp\file2.txt SUCCESS 1:21:29.0102701 AM test.exe 6132 WriteFile C:\temp\file2.txt SUCCESS Offset: 0, Length: 4,096, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Priority: Normal 1:21:29.0113444 AM test.exe 6132 WriteFile C:\temp\file2.txt SUCCESS Offset: 72, Length: 24 1:21:29.0113602 AM test.exe 6132 FlushBuffersFile C:\temp\file2.txt SUCCESS 1:21:29.0113848 AM test.exe 6132 WriteFile C:\temp\file2.txt SUCCESS Offset: 0, Length: 4,096, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Priority: NormalSo the reason that your benchmark shows a severe disadvantage to the
WriteFile()loop is simply because you have about a thousand calls toFlushFileBuffers()that aren't in thefwrite()loop.这篇关于在windows中fwrite比WriteFile快吗？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
