最好的内存缓冲区大小 - 与流文件I / O [英] File I/O with streams - best memory buffer size

问题描述

我写一个小的I / O库，以协助较大（爱好）项目。这个库的一部分执行通过的FileStream 对象写在一个文件，该文件读取各种功能/。在每个 StreamReader.Read（...）通，

I am writing a small I/O library to assist with a larger (hobby) project. A part of this library performs various functions on a file, which is read / written via the FileStream object. On each StreamReader.Read(...) pass,

我火了，这将在主应用程序可用于显示进度信息的事件。在循环的推移的处理是vaired，但不是太耗费时间（它可能只是一个简单的文件副本，例如，或者可涉及加密...）。

I fire off an event which will be used in the main app to display progress information. The processing that goes on in the loop is vaired, but is not too time consuming (it could just be a simple file copy, for example, or may involve encryption...).

我的主要问题是：什么是用最好的内存缓冲区的大小？有关物理磁盘布局思考，我可以挑选2K，这将覆盖CD扇区大小，是一个512字节的硬盘扇区的一个很好的倍数。越往上抽象的树，你可以去它可以一次读取整个FAT簇更大的缓冲区。我知道今天的电脑，我就可以去了更多的内存选项饿（一对夫妇的MIB，例如），但后来我增加UI更新和用户感知一个不太敏感的应用程序之间的时间。

My main question is: What is the best memory buffer size to use? Thinking about physical disk layouts, I could pick 2k, which would cover a CD sector size and is a nice multiple of a 512 bytes hard disk sector. Higher up the abstraction tree, you could go for a larger buffer which could read an entire FAT cluster at a time. I realise with today's PC's, I could go for a more memory hungry option (a couple of MiB, for example), but then I increase the time between UI updates and the user perceives a less responsive application.

一边作为，我最终还是希望能提供类似的接口托管于FTP / HTTP服务器上的文件（通过局域网/ fastish DSL）。什么是对那些最好的内存缓冲区大小（同样，感知响应速度与性能之间的最佳情况的权衡）？

As an aside, I'm eventually hoping to provide a similar interface to files hosted on FTP / HTTP servers (over a local network / fastish DSL). What would be the best memory buffer size for those (again, a "best-case" tradeoff between perceived responsiveness vs. performance)?

推荐答案

文件由文件系统缓存已经缓冲。你只需要选择一个缓冲区大小不强迫的FileStream，使本机Windows的ReadFile（）API调用来填充缓冲区过于频繁。不要去千字节以下，超过16 KB是浪费内存和不友好的CPU的 L1缓存。（通常为16或32＆NBSP; KB的数据）

Files are already buffered by the file system cache. You just need to pick a buffer size that doesn't force FileStream to make the native Windows ReadFile() API call to fill the buffer too often. Don't go below a kilobyte, more than 16 KB is a waste of memory and unfriendly to the CPU's L1 cache (typically 16 or 32 KB of data).

4 NBSP; KB是一个传统的选择，尽管这将完全按事故跨越虚拟内存页面只有永远。是困难的轮廓;你会最终测量需要多长时间来读取缓存的文件。它运行在RAM的速度下，5千兆字节/秒和最多，如果数据是在高速缓存中可用。这将是在缓存中运行测试的第二次，并且不会在生产环境中发生的太频繁。文件I / O是完全由磁盘驱动器或 NIC为主以及冰川是缓慢的，复制的数据是花生。 4＆NBSP; KB将正常工作

4 KB is a traditional choice, even though that will exactly span a virtual memory page only ever by accident. It is difficult to profile; you'll end up measuring how long it takes to read a cached file. Which runs at RAM speeds, 5 gigabytes/sec and up if the data is available in the cache. It will be in the cache the second time you run your test, and that won't happen in a production environment too often. File I/O is completely dominated by the disk drive or the NIC and is glacially slow, copying the data is peanuts. 4 KB will work fine.

这篇关于最好的内存缓冲区大小 - 与流文件I / O的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！