为什么移动缓冲区指针减慢FREAD（C编程语言）？ [英] Why does moving the buffer pointer slow down fread (C programming language)?
问题描述
我使用读1 GB的文件的 FREAD 在C.我读在1MB块的文件，使用以下循环：
I am reading a 1 GB file using fread in C. I am reading the file in 1MB chunks, using the following loop:
FILE *fp;
fp = fopen(filename, "rb");
unsigned char* buf;
buf = malloc(CHUNK_SIZE);
for(i = 0; i < NUMBER_OF_CHUNKS; ++i)
{
fread(buf, CHUNK_SIZE, 1, fp);
//Do something with contents of buffer
}
fclose(fp);
阅读这种方式，大约需要2秒钟的文件。
Reading the file this way takes ~2 seconds.
不过，我决定，我想分配一个大的缓冲整个文件的内容，而不是在每次迭代移动缓冲区指针在 FREAD 函数里，像这样的：
However, I decided that I wanted to allocate one big buffer for the contents of the whole file instead and "move the buffer pointer" inside the fread function at each iteration, like this:
FILE *fp;
fp = fopen(filename, "rb");
unsigned char* buf;
buf = malloc(CHUNK_SIZE * NUMBER_OF_CHUNKS);
for(i = 0; i < NUMBER_OF_CHUNKS; ++i)
{
fread(&buf[i*CHUNK_SIZE], CHUNK_SIZE, 1, fp);
}
fclose(fp);
这会减慢阅读显著，现在大约需要〜40秒。
This slows down the reading significantly, it now takes about ~40 seconds.
我的问题是：
- 为什么这对性能？如此巨大的影响
- 什么你会建议我做，如果我想读的第二种方式的文件，但我想保持时间低？
该文件包含字母数字字符的单行。
The file consists of a single line of alphanumeric characters.
我想在第二方式读取它，这样我可以有其他线程访问那些已经读入缓冲器中的数据块，而读取线程继续填充缓冲区的其余部分。
I want to read it in the second way, so that I can have other threads access the chunks in the buffer that are already read, while the reading thread continues filling the rest of the buffer.
感谢您！
推荐答案
这是可能的，你是你的机器上运行内存。 1千兆是相当多的内存来分配。您的操作系统我有交换一些数据到磁盘上，这将导致经济放缓幅度的秩序。
It's possible that you are running out of memory on your machine. A gigabyte is rather a lot of memory to allocate. Your OS my have to swap some of the data to disk, which will cause an order of magnitude slowdown.
您可以考虑单独分配每个块，并释放他们时，他们与完成。这样，你的程序的总内存使用量由工作集为界，而不是整个文件。
You could consider allocating each chunk individually, and freeing them when they are done with. This way the total memory usage of your program is bounded by the working set, rather than the entire file.
这篇关于为什么移动缓冲区指针减慢FREAD（C编程语言）？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
