与Read()相比，FileStream.ReadAsync非常慢 [英] FileStream.ReadAsync very slow compared to Read()

问题描述

我有以下代码循环遍历一个文件并一次读取1024个字节.第一次迭代使用 FileStream.Read()，第二次迭代使用 FileStream.ReadAsync().

I have the following code to loop thru a file and read 1024 bytes at a time. The first iteration uses FileStream.Read() and the second iteration uses FileStream.ReadAsync().

private async void Button_Click(object sender, RoutedEventArgs e)
{
    await Task.Run(() => Test()).ConfigureAwait(false);
}

private async Task Test()
{
    Stopwatch sw = new Stopwatch();
    sw.Start();

    int readSize;
    int blockSize = 1024;
    byte[] data = new byte[blockSize];

    string theFile = @"C:\test.mp4";
    long totalRead = 0;

    using (FileStream fs = new FileStream(theFile, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
    {

        readSize = fs.Read(data, 0, blockSize);

        while (readSize > 0)
        {
            totalRead += readSize;
            readSize = fs.Read(data, 0, blockSize);
        }
    }

    sw.Stop();
    Console.WriteLine($"Read() Took {sw.ElapsedMilliseconds}ms and totalRead: {totalRead}");

    sw.Reset();
    sw.Start();
    totalRead = 0;
    using (FileStream fs = new FileStream(theFile, FileMode.Open, FileAccess.Read, FileShare.ReadWrite, (blockSize*2), FileOptions.Asynchronous | FileOptions.SequentialScan))
    {
        readSize = await fs.ReadAsync(data, 0, blockSize).ConfigureAwait(false);

        while (readSize > 0)
        {
            totalRead += readSize;
            readSize = await fs.ReadAsync(data, 0, blockSize).ConfigureAwait(false);
        }
    }

    sw.Stop();
    Console.WriteLine($"ReadAsync() Took {sw.ElapsedMilliseconds}ms and totalRead: {totalRead}");
}

结果:

Read() Took 162ms and totalRead: 162835040
ReadAsync() Took 15597ms and totalRead: 162835040

ReadAsync()慢大约100倍.我有什么想念的吗?我唯一能想到的是使用ReadAsync()创建和销毁任务的开销，但是开销那么大吗?

The ReadAsync() is about 100 times slower. Am I missing anything? The only thing I can think of is the overhead to create and destroy task using ReadAsync(), but is the overhead that much?

更新:

我已经更改了上面的代码以反映@Cory的建议.有一点改进:

I've changed the above code to reflect the suggestion by @Cory. There is a slight improvement:

Read() Took 142ms and totalRead: 162835040 
ReadAsync() Took 12288ms and totalRead: 162835040

当我按照@Alexandru的建议将读取的块大小增加到1MB时，结果会更容易接受:

When I increase the read block size to 1MB as suggested by @Alexandru, the results are much more acceptable:

Read() Took 32ms and totalRead: 162835040
ReadAsync() Took 76ms and totalRead: 162835040

因此，它向我暗示，确实是导致缓慢的任务数量的开销.但是，如果任务的创建和销毁仅需100µs，那么以小块大小的速度仍然无法真正解决问题.

So, it hinted to me that it is indeed the overhead of the number of tasks which causes the slowness. But, if the creation and destroy of task only takes merely 100µs, things still don't really adds up for the slowness with a small block size.

推荐答案

如果要进行异步操作，请使用大缓冲区，并确保在FileStream构造函数中打开异步模式，并且您应该没事.这样等待的异步方法将在当前线程中进出(请注意，当前线程是您所在的情况下的UI线程，可以由促进该方法进出线程的任何其他异步方法所滞后)，因此如果您有大量的调用(例如，调用一个新的线程构造函数并等待其完成大约10万次)，则此过程将涉及一些开销，尤其是当您要处理需要等待的UI应用程序时UI线程是空闲的，以便在异步功能完成后捕获到其中).因此，为了减少这些调用，我们只需以更大的增量读取数据，然后通过增加缓冲区大小将应用程序集中在一次读取更多数据上.确保在发布模式下测试此代码，以便我们可以使用所有编译器优化，并且还可以使调试器不会减慢我们的速度:

Stick with big buffers if you're doing async and make sure to turn on async mode in the FileStream constructor, and you should be okay. Async methods that you await like this will trap in and out of the current thread (mind you the current thread is the UI thread in your case, which can be lagged by any other async method facilitating the same in and out thread trapping) and so there will be some overhead involved in this process if you have a large number of calls (imagine calling a new thread constructor and awaiting for it to finish about 100K times, and especially if you're dealing with a UI app where you need to wait for the UI thread to be free in order to trap back into it once the async function completes). So, to reduce these calls, we simply read in larger increments of data and focus the application on reading more data at a time by increasing the buffer size. Make sure to test this code in Release mode so all of the compiler optimizations are available to us and also such that the debugger does not slow us down:

class Program
{
    static void Main(string[] args)
    {
        DoStuff();
        Console.ReadLine();
    }

    public static async void DoStuff()
    {
        var filename = @"C:\Example.txt";

        var sw = new Stopwatch();
        sw.Start();
        ReadAllFile(filename);
        sw.Stop();

        Console.WriteLine("Sync: " + sw.Elapsed);

        sw.Restart();
        await ReadAllFileAsync(filename);
        sw.Stop();

        Console.WriteLine("Async: " + sw.Elapsed);
    }

    static void ReadAllFile(string filename)
    {
        byte[] buffer = new byte[131072];
        using (var file = new FileStream(filename, FileMode.Open, FileAccess.Read, FileShare.Read, buffer.Length, false))
            while (true)
                if (file.Read(buffer, 0, buffer.Length) <= 0)
                    break;
    }

    static async Task ReadAllFileAsync(string filename)
    {
        byte[] buffer = new byte[131072];
        using (var file = new FileStream(filename, FileMode.Open, FileAccess.Read, FileShare.Read, buffer.Length, true))
            while (true)
                if ((await file.ReadAsync(buffer, 0, buffer.Length)) <= 0)
                    break;
    }
}

结果:

同步:00:00:00.3092809

Sync: 00:00:00.3092809

异步:00:00:00.5541262

Async: 00:00:00.5541262

几乎可以忽略不计...该文件约为1 GB.

Pretty negligible...the file is about 1 GB.

比方说，我要更大一些，一个1 MB的缓冲区，又名新字节[1048576] (来吧，这天，每个人都有1 MB的RAM):

Let's say I go even bigger, a 1 MB buffer, AKA new byte[1048576] (come on man, everyone has 1 MB of RAM these days):

同步:00:00:00.2925763

Sync: 00:00:00.2925763

异步:00:00:00.3402034

Async: 00:00:00.3402034

然后只有几分之一秒的差异.如果眨眼，就会错过它.

Then its just a few hundredths of a second difference. If you blink, you'll miss it.

这篇关于与Read()相比，FileStream.ReadAsync非常慢的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！