最佳缓冲区大小为阅读过程写 [英] Optimal Buffer size for read-process-write

问题描述

在我的功能，我需要从一个文件中的一些数据读入缓冲区，处理这些数据，并将其写回到另一个文件。该文件是未知大小的并且可能是非常大的。

In my function, I need to read some data from a file into a buffer, manipulate the data and write it back to another file. The file is of unknown size and may be very large.

如果我用一个小缓冲器，会有很长的读/写周期，这将花费太多时间。相比之下，长缓冲意味着我需要消耗更多的内存。什么是我应该使用最佳的缓冲区大小？这箱子是依赖？

If I use a small buffer, there will be a long read/write cycle and it would take much time. In contrast, long buffer means I need to consume more memory. What is the optimal buffer size I should use? Is this case dependent?

我看到像'万亿复制窗口中的某些应用能够有效管理庞大的文件。是否有任何其他技术或机制，我应该知道的？

I saw some application like 'Tera copy' in windows that manages huge files efficiently. Is there any other technique or mechanism I should be aware of?

请注意：这个程序将在Windows下运行

Note: This program will be running under Windows.

推荐答案

请参阅微软不得不说的IO大小：<一href=\"http://technet.microsoft.com/en-us/library/cc938632.aspx\">http://technet.microsoft.com/en-us/library/cc938632.aspx.基本上，他们说你应该做的IO在64K块。

See what Microsoft has to say about IO size: http://technet.microsoft.com/en-us/library/cc938632.aspx. Basically, they say you should probably do IO in 64K blocks.

在* NIX平台上， struct stat中有一个 st_blksize 部件，其所说的话应该是最小的IO块大小

On *NIX platforms, struct stat has a st_blksize member which says what should be a minimal IO block size.

这篇关于最佳缓冲区大小为阅读过程写的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！