什么是读取/写入到磁盘中的.NET的最快方法？ [英] What's the fastest way to read/write to disk in .NET?
问题描述
我有一个读取和磁盘写入文件的小程序。其分解到最简单的层面，它从一个文件流读取字节，并将其写入到另一个。它履行职责精细，但它并不是最快的事。
I've got a little program that reads and writes files on disk. Breaking it down to the most simple level, it reads bytes from one file stream and writes them to another. It performs its duties fine, but it isn't the fastest thing.
我已经看到了，可以通过千兆字节或撕裂更多的读取其他应用程序/写的惊人速度。显然，他们正在操作更接近金属造成不小的.NET应用程序。
I've seen other applications that can tear through a gigabyte or more of reads/writes in amazing speeds. Obviously they're operating closer to the metal than a little .NET app.
什么是最有效的.NET API，用于从磁盘分流到/？什么的Win32 API可（和价值P /调用的）的快速磁盘访问？
What are the most efficient .NET APIs for streaming to/from the disk? What win32 APIs are available (and worth p/invoking for) for speedy disk access?
推荐答案
快速文件I / O是少谈具体的API调用你做，而是你如何构建应用程序的I / O工作。
Fast file I/O is less about the specific API calls you make, but rather about how you architect your application to work with I/O.
如果您在单个线程以连续的方式进行所有的I / O操作的，例如：
If you are performing all of your I/O operations on a single thread in a sequential manner, for example
- 在读取块到内存
- 在存储过程块莫名其妙地
- 写块出文件
- 在重复，直到完成...
正在瓶颈在单个线程的处理的循环系统的I / O带宽。另一种，但更复杂的设计是多线程应用程序，以最大限度地提高吞吐量，并避免等待时间。这使得该系统能够同时利用两者的CPU和I / O控制器的带宽。一个典型的设计，这看起来是这样的：
you are bottlenecking the system's I/O bandwidth in the processing loop of a single thread. An alternative, but more complicated design is to multithread your application to maximize throughput and avoid wait time. This allows the system to take advantage of both CPU and I/O controller bandwidth simultaneously. A typical design for this would look something like:
- 在一个（或多个）的工作线程从磁盘读取数据并将其添加到共享输入队列
- 在一个（或多个）的工作线程读取共享输入队列块，处理它们，并将它们添加到共享输出队列
- 在一个（或多个）的工作线程读取经过加工阻止共享输出队列，并将其写入到相应的输出文件。
这是并不是一件容易的架构来设计权，并且需要相当多的思想，以避免造成内存锁争用，或者抑制了系统的并发I / O请求。还需要提供控制元数据，以便输出处理的状态上没有一个线程的调用堆栈而是在输入/输出工作队列进行管理。你还必须确保你变换和写在正确的顺序输出，因为多线程I / O你不能确定的工作被放置在输入队列中的一个保证的顺序。它是复杂的 - 但它是可能的，而且它可以具有在串行方法中通过一个显着的差异
This is not an easy architecture to design right, and requires quite a bit of thought to avoid creating in-memory lock contention, or overwhelm the system with concurrent I/O requests. You also need to provide control metadata so that the state of output processing is not managed on the call stack of a thread but rather in the input/output work queues. You also have to make sure that you transform and write the output in the correct order, since with multi-threaded I/O you can't be sure work is placed on the input queue in a guaranteed order. It's complicated - but it is possible, and it can have a dramatic difference in throughput over a serial approach.
如果你确实有时间和想挤进每盎司性能从系统中，你也可以使用的 I / O完成端口 - 一个比较低层次的API - 以最大化吞吐量
If you really have the time and want to squeeze every ounce of performance from the system, you could also use I/O completion ports - a relatively low-level API - to maximize throughput.
祝你好运。
这篇关于什么是读取/写入到磁盘中的.NET的最快方法？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
