如何充分利用磁盘IO排队 [英] How to take advantage of Disk IO queueing

问题描述

我需要从一个3.7 GB的文件中读取数据的小序列。我需要阅读的位置是的不相邻，但使该文件从头到尾读我可以订购IO。

I need to read small sequences of data from a 3.7 GB file. The positions I need to read are not adjacent, but I can order the IO so that the file is read from beginning to end.

文件存储在一个iSCSI SAN中应该是能够处理/优化排队IO。

The file is stored on a iSCSI SAN which should be capable of handling/optimizing queued IO.

现在的问题是，如何才能让所有的数据的一杆请求/位置我需要一气呵成？是否可以？我不认为异步IO是一个选项，因为阅读是非常小的（20-200字节）

The question is, how can I make a one shot request of all the data/positions I need in one go? Is it possible? I don't think async IO is an option because the reads are very small (20-200 bytes)

目前的代码如下所示：

using (var fileStream = new FileStream(dataStorePath, FileMode.Open, FileAccess.Read, FileShare.Read))
{
    for (int i = 0; i < internalIds.Count();i++ )
    {
        fileStream.Position = seekPositions[i].SeekPosition;
        ... = Serializer.DeserializeWithLengthPrefix<...>(fileStream, PrefixStyle.Base128);

    }
    ...
}

我在寻找各种方法来改善这种I / O，因为我越来越有点低于平均水平的读取性能。从移动磁头似乎加入了所有的寻道时间

I'm looking for ways to improve this I/O because I'm getting somewhat sub-par read performance. All the seek times from moving the head seem to be adding up.

推荐答案

你有没有运行的性能监视器（来自微软的Sysinternals）？

Have you run Performance Monitor (from Microsoft Sysinternals) on this?

我不知道是什么问题，但我会采取一种猜测。如果你从SAN读书，我想磁盘访问导致引擎盖下的网络请求。第一读取发送寻求请求，读取和缓冲器数据，然后将该串行构造的对象。通过你的第二个请求被发送的时间，SAN磁盘继续旋转，所以你必须等待数据旋转到位。

I'm not sure what the problem is, but I'll take a guess. If you're reading from a SAN, I would think disk accesses result in network requests under the hood. The first read sends a request to seek, reads and buffers data, and then the Serializer constructs the objects. By the time your second request gets sent, the SAN disks have continued to spin, so you have to wait for the data to spin into place.

你试过多线程？我很好奇的表现，如果你安装文件部分的队列，你需要按顺序来处理，旋转了某些线程，让他们单独打开的文件（FileSharing.Read所以它们都可以访问该文件一次）和然后让他们开始从队列抓的工作。结果输出到另一个集合。如果为了使输出事宜，你按在你排队它们原来的顺序输出。

Have you tried multithreading? I'm curious about the performance if you setup a Queue of file sections you need to process in sequential order, spin up a some threads, have them open the file separately (FileSharing.Read so they can all access the file at once) and then let them start grabbing work from the Queue. Output the results into another collection. If the order matters for the output, you sort the output by the original order in which you queued them.

---编辑---

您是否尝试过的 ReadFileScatter API ？ 下面是从P-调用签名pinvoke.net 。

这篇关于如何充分利用磁盘IO排队的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！