优化文件缓冲区读取大小？ [英] Optimum file buffer read size?

问题描述

我在写这需要读取相当大的文件的应用程序。我一直想知道什么是一个现代化的Windows XP计算机上读取缓冲区的最佳尺寸。我用Google搜索，发现了很多例子其中有1024的最佳尺寸。

I am writing an application which needs to read fairly large files. I have always wondered what's the optimum size for the read buffer on a modern Windows XP computer. I googled and found many examples which had 1024 as the optimum size.

下面是我的意思的一个片段：

Here is a snippet of what I mean:

long pointer = 0;
buffer = new byte[1024]; // What's a good size here ?
while (pointer < input.Length)
{
    pointer += input.Read(buffer, 0, buffer.Length);
}

我的应用程序相当简单，所以我不打算写任何基准测试code，但想知道什么大小很常见？

My application is fairly simple, so I am not looking to write any benchmarking code, but would like to know what sizes are common?

推荐答案

1k的缓冲大小似乎小了一点。一般情况下，不存在一刀切的缓冲区大小。您需要设置一个适合你的算法的行为缓冲区大小。现在，一般，它不是一个好主意，有一个非常庞大的缓冲器，但是，有一个是太小或不符合你如何处理每块不是很大要么。

A 1k buffer size seems a bit small. Generally, there is no "one size fits all" buffer size. You need to set a buffer size that fits the behavior of your algorithm. Now, generally, its not a good idea to have a really huge buffer, but, having one that is too small or not in line with how you process each chunk is not that great either.

如果你只是又一个读取数据的一个块全部到内存中处理之前，我会用一个更大的缓冲区。我可能会使用8K或16K，但可能不会大。

If you are simply reading data one chunk after another entirely into memory before processing it, I would use a larger buffer. I would probably use 8k or 16k, but probably not larger.

在另一方面，如果要处理的流的方式，在读块然后读取下前处理它的数据，更小的缓冲器可能更为有用。更妙的是，如果你是数据流，有结构，我会改变阅读专门匹配的数据，你正在阅读的类型的数据量。例如，如果你正在阅读一个包含4个字符code，花车，和一个字符串二进制数据，我会读了4个字符code到4字节数组，以及浮。我会读的字符串的长度，然后创建一个缓冲区读取字符串数据的整个块一次。

On the other hand, if you are processing the data in streaming fashion, reading a chunk then processing it before reading the next, smaller buffers might be more useful. Even better, if you are streaming data that has structure, I would change the amount of data read to specifically match the type of data you are reading. For example, if you are reading binary data that contains a 4-character code, a float, and a string, I would read the 4-character code into a 4-byte array, as well as the float. I would read the length of the string, then create a buffer to read the whole chunk of string data at once.

如果你正在做的流数据处理，我将调查和BinaryReader在类的BinaryWriter。这允许你用二进制数据很容易的工作，而不必过多地担心数据本身。它也可以让你从分离您正在使用实际数据的缓冲区大小。你可以用轻松的设置BinaryReader在一个16K的缓冲底层的流上，并读取单个数据值。

If you are doing streaming data processing, I would look into the BinaryReader and BinaryWriter classes. These allow you to work with binary data very easily, without having to worry much about the data itself. It also allows you to decouple your buffer sized from the actual data you are working with. You could set a 16k buffer on the underlying stream, and read individual data values with the BinaryReader with ease.

这篇关于优化文件缓冲区读取大小？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！