无缓冲输出非常慢 [英] Unbuffered Output Very Slow

问题描述

我生成使用概述的方法的数据库非常大的.csv文件

I generate a very large .csv file from a database using the method outlined in

http://stackoverflow.com/a/13456219/141172

它工作正常，到一个点。当导出的文件太大，我得到一个 OutOfMemoryException异常。

It works fine, up to a point. When the exported file is too large, I get an OutOfMemoryException.

如果我通过修改code这样的关闭输出缓冲：

If I turn off output buffering by modifying that code like this:

protected override void WriteFile(System.Web.HttpResponseBase response)
{
    response.BufferOutput = false; // <--- Added this
    this.Content(response.OutputStream);
}

文件下载完成。然而，这是几级的订单的时候输出缓冲启用（用于缓冲真/假同一个文件测量，在localhost）慢。

the file download completes. However, it is several orders of magnitude slower than when output buffering was enabled (measured for the same file with buffering true/false, on localhost).

我的理解是比较慢的，但为什么会放缓至一个相对的抓取？有什么我可以做，以提高处理速度？

I understand that is slower, but why would it slow to a relative crawl? Is there anything I can do to improve processing speed?

更新

这也将是使用文件的选项（流流，字符串的contentType）作为意见提出。但是，我不知道如何创建流。该数据是根据一个DB查询动态组合，和一个MemoryStream将耗尽的连续的物理存储器。建议都欢迎。

It would also be an option to use File(Stream stream, String contentType) as suggested in the comments. However, I'm not sure how to create stream. The data is dynamically assembled based on a DB query, and a MemoryStream would run out of contiguous physical memory. Suggestions are welcome.

更新2

有人建议在交替地从所述数据库读出和写入到流引起的劣化的评论。我修改了code在一个单独的线程执行流写入（使用生产者/消费者模式）。在性能没有AP preciable差异。

It was suggested in the comments that alternately reading from the database and writing to the stream is causing a degradation. I modified the code to perform the stream writing in a separate thread (using the producer/consumer pattern). There is no appreciable difference in performance.

推荐答案

我不知道是什么ASP.NET和IIS与输出流做的正是，但也许是太小块正在使用。胡克在 BufferedStream 有了非常大的缓冲区中，类似4MB。

I don't know what ASP.NET and IIS are doing exactly with output streaming but maybe too small chunks are being uses. Hook in a BufferedStream with a very big buffer, like 4MB.

根据您的意见，它的工作。现在，调低缓冲区大小，以节省存储空间，拥有更小的工作组。适合缓存。

According to your comments it worked. Now, tune down the buffer size to save memory and have a smaller working set. Good for cache.

作为一个主观的评论我很失望，这是的必要。 IIS会自动使用正确的缓冲区这与TCP连接非常容易的。

As a subjective comment I'm disappointed that this is even necessary. IIS should use the right buffers automatically which is extremely easy with TCP connections.

编辑从OP

下面是这个答案得到的code

Here is the code derived from this answer

public ActionResult Export()
{
    // Domain specific stuff here
    return new FileGeneratingResult("MyFile.txt", "text/text",
            stream => this.StreamExport(stream), false);
}

private void StreamExport(Stream stream)
{
    using (BufferedStream bs = new BufferedStream(stream, 256*1024))
    using (StreamWriter sw = new StreamWriter(bs))  
    foreach (var stuff in MyData())
    {
        sw.Write(stuff);
    }
}

这篇关于无缓冲输出非常慢的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！