异步文件读取比同步或手动线程慢40倍 [英] async file reading 40 times slower than synchronous or manual Threads

问题描述

我有3个文件，每个文件长1百万行，我正在逐行阅读它们.没有处理，只是在阅读，因为我只是在试用.

I have 3 files, each 1 million rows long and I'm reading them line by line. No processing, just reading as I'm just trialling things out.

如果我同步执行此操作，则需要1秒钟.如果我改用Threads(每个文件一个)，则速度会稍微快一些(代码不在下面，但我只是创建了一个新的Thread并为每个文件启动了它).

If I do this synchronously it takes 1 second. If I switch to using Threads, one for each file, it is slightly quicker (code not below, but I simply created a new Thread and started it for each file).

当我更改为异步时，它的耗时是40秒的40倍.如果我添加任何工作来进行实际处理，则无法看到如何在同步上使用异步，或者我是否想要使用线程的响应式应用程序.

When I change to async, it is taking 40 times as long at 40 seconds. If I add in any work to do actual processing, I cannot see how I'd ever use async over synchronous or if I wanted a responsive application using Threads.

还是我在这段代码中做了根本上错误的事情，而不是异步的意图?

Or am I doing something fundamentally wrong with this code and not as async was intended?

谢谢.

class AsyncTestIOBound
{
    Stopwatch sw = new Stopwatch();
    internal void Tests()
    {
        DoSynchronous();
        DoASynchronous();
    }
    #region sync
    private void DoSynchronous()
    {
        sw.Restart();
        var start = sw.ElapsedMilliseconds;
        Console.WriteLine($"Starting Sync Test");
        DoSync("Addresses", "SampleLargeFile1.txt");
        DoSync("routes   ", "SampleLargeFile2.txt");
        DoSync("Equipment", "SampleLargeFile3.txt");
        sw.Stop();
        Console.WriteLine($"Ended Sync Test. Took {(sw.ElapsedMilliseconds - start)} mseconds");
        Console.ReadKey();
    }

    private long DoSync(string v, string filename)
    {
        string line;
        long counter = 0;
        using (StreamReader file = new StreamReader(filename))
        {
            while ((line = file.ReadLine()) != null)
            {
                counter++;
            }
        }
        Console.WriteLine($"{v}: T{Thread.CurrentThread.ManagedThreadId}: Lines: {counter}");
        return counter;
    }
    #endregion

    #region async
    private void DoASynchronous()
    {
        sw.Restart();
        var start = sw.ElapsedMilliseconds;
        Console.WriteLine($"Starting Sync Test");
        Task a=DoASync("Addresses", "SampleLargeFile1.txt");
        Task b=DoASync("routes   ", "SampleLargeFile2.txt");
        Task c=DoASync("Equipment", "SampleLargeFile3.txt");
        Task.WaitAll(a, b, c);
        sw.Stop();
        Console.WriteLine($"Ended Sync Test. Took {(sw.ElapsedMilliseconds - start)} mseconds");
        Console.ReadKey();
    }

    private async Task<long> DoASync(string v, string filename)
    {
        string line;
        long counter = 0;
        using (StreamReader file = new StreamReader(filename))
        {
            while ((line = await file.ReadLineAsync()) != null)
            {
                counter++;
            }
        }
        Console.WriteLine($"{v}: T{Thread.CurrentThread.ManagedThreadId}: Lines: {counter}");
        return counter;
    }
    #endregion

}

推荐答案

由于您在巨型循环中多次使用了await(在您的情况下，循环遍历"SampleLargeFile"的每一行)，因此您正在执行很多上下文切换，开销可能真的很糟糕.

Since you are using await several times in a giant loop (in your case, looping through each line of a "SampleLargeFile"), you are doing a lot of context switching, and the overhead can be really bad.

对于每一行，您的代码可能正在每个文件之间切换.如果您的计算机使用硬盘驱动器，则情况可能更糟.想象一下HD的头部变得疯狂.

For each line, your code maybe is switching between each file. If your computer uses a hard drive, this can get even worse. Imagine the head of your HD getting crazy.

使用普通线程时，您不会切换每行的上下文.

When you use normal threads, you are not switching the context for each line.

要解决此问题，只需一次运行即可读取文件.您仍然可以使用async/await(ReadToEndAsync())并获得良好的性能.

To solve this, just read the file on a single run. You can still use async/await (ReadToEndAsync()) and get a good performance.

编辑

因此，您正在尝试使用异步计数文本文件上的行，对吧?

So, you are trying to count lines on the text file using async, right?

尝试此操作(无需将整个文件加载到内存中):

Try this (no need to load the entire file in memory):

private async Task<int> CountLines(string path)
{
    int count = 0;
    await Task.Run(() =>
    {
        using (FileStream fs = File.Open(path, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
        using (BufferedStream bs = new BufferedStream(fs))
        using (StreamReader sr = new StreamReader(bs))
        {
            while (sr.ReadLine() != null)
            {
                count++;
            }
        }
    });
    return count;
}

这篇关于异步文件读取比同步或手动线程慢40倍的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！