WebAPI 请求流支持 [英] WebAPI Request Streaming support

问题描述

我正在编写一个 ASP.NET Web API 应用程序，它要求我接受文件上传并将该文件转发到另一个 HTTP 端点.

I am writing an ASP.NET Web API application that requires me to accept a file upload and forward that file on to another HTTP endpoint.

我担心，如果许多用户尝试上传一个 100MB 的文件(这是一个有效的用例)，那么我的应用程序将占用大量内存，并且根据大请求的数量，这个占用空间可能会变得很大，而我的应用程序会倒塌并死亡.

I am concerned that if many users try to upload a 100MB file each (which is a valid use case) then my application will have a large memory footprint and depending on the volume of large requests this footprint could grow to large and my application would keel over and die.

理想情况下，我希望在 Web 服务器开始接收文件后立即开始将文件流式传输到另一个 HTTP 端点，以显着减少服务器上的负载.

Ideally I'd like begin streaming the file to the other HTTP end point as soon as the web server starts to receive the file to significantly reduce the load on the server.

我确定这个过程有一个名字，但我不知道 - 这使得寻找它变得相当困难.

I'm sure this process has a name but I don't know it - which is making searching for it rather tough.

我在 Web API 中使用响应流做了很多工作，但我以前从未考虑过请求流.

I've done quite a bit of work with Response Streaming in the Web API but I've never had to consider request streaming before.

我能告诉你的最好方法是:

Best I can tell I need to work out how to:

• 在完成上传之前开始处理流.
• 使用 HttpClient 将相同的请求流式传输到另一个 HTTP 端点.

谁能给我一些指点?

推荐答案

这是一个有趣的问题.我会尽力提供一些一般性的建议.

That's an interesting question. I'll try to do my best to give some general pointers.

需要考虑的几点:

1) 默认情况下，Web API 会缓冲请求，因此您担心内存占用量可能很大，这绝对是有道理的.您可以强制 Web API 以流式模式处理请求:

1) Web API by default buffers requests so your fear that the memory footprint might be considerable is definitely justified. You can force Web API to work with requests in a streamed mode:

    public class NoBufferPolicySelector : WebHostBufferPolicySelector
    {
       public override bool UseBufferedInputStream(object hostContext)
       {
          var context = hostContext as HttpContextBase;

          if (context != null)
          {
             if (string.Equals(context.Request.RequestContext.RouteData.Values["controller"].ToString(), "uploading", StringComparison.InvariantCultureIgnoreCase))
                return false;
          }

          return true;
       }

       public override bool UseBufferedOutputStream(HttpResponseMessage response)
       {
          return base.UseBufferedOutputStream(response);
       }
    }

然后更换服务:

GlobalConfiguration.Configuration.Services.Replace(typeof(IHostBufferPolicySelector), new NoBufferPolicySelector());

请注意，由于此时 WebHost 和 SelfHost 之间的差异，此类更改只能在 WebHost 中进行.如果您的端点是自托管的，则必须在 GlobalConfig 级别设置流模式:

Please note that due to differences between WebHost and SelfHost at this point, such change is only possible in WebHost. If your endpoint is selfHosted, you would have to set the streaming mode at the GlobalConfig level:

//requests only
selfHostConf.TransferMode = TransferMode.StreamedRequest;
//responses only
selfHostConf.TransferMode = TransferMode.StreamedResponse;
//both
selfHostConf.TransferMode = TransferMode.Streamed;

我之前写过关于在 Web API 中更详细地处理大文件的博客 - http://www.strathweb.com/2012/09/dealing-with-large-files-in-asp-net-web-api/ 所以希望你会觉得这很有用.

I have blogged about dealing with large files in Web API in more details before - http://www.strathweb.com/2012/09/dealing-with-large-files-in-asp-net-web-api/ so hopefully you'll find that useful.

2) 其次，如果你使用 HttpClient，在 .NET 4 中它默认缓冲请求体，所以你真的应该使用 .NEt 4.5.

2) Secondly, if you use HttpClient, in .NET 4 it buffers the requests body by default, so you should really use .NEt 4.5.

如果您必须使用 .NET 4，则必须直接使用 HttWebRequest:- http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.allowreadstreambuffering.aspx- http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.allowwritestreambuffering.aspx

If you have to use .NET 4 you have to work with HttWebRequest directly: - http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.allowreadstreambuffering.aspx - http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.allowwritestreambuffering.aspx

3) 至于将数据推送到客户端，如果您想这样做，绝对有可能，使用 PushStreamContent.Henrik 有一篇简短的介绍性文章 - http://blogs.msdn.com/b/henrikn/archive/2012/04/23/using-cookies-with-asp-net-web-api.aspx(这是基于 Web API RC 位，因此您可能需要调整一些签名等)我还在博客上写了关于在此处推送大块流数据 - http://www.strathweb.com/2013/01/asynchronously-streaming-video-with-asp-net-web-api/

3) As far as pushing the data to the client that's definitely possible if you want to do that, with PushStreamContent. Henrik has a short introductory post here - http://blogs.msdn.com/b/henrikn/archive/2012/04/23/using-cookies-with-asp-net-web-api.aspx (it's based on Web API RC bits so you might need to adjust some signatures etc.) I also blogged about pushing chunks of stream data here - http://www.strathweb.com/2013/01/asynchronously-streaming-video-with-asp-net-web-api/

要查看请求中 PushStreamContent 的示例，您可以查看此示例解决方案 - http://aspnet.codeplex.com/SourceControl/changeset/view/bb167f0b0013#Samples/Net45/CS/WebApi/UploadXDocumentSample/ReadMe.txt

To see an example if PushStreamContent in the request, you can have a look at this sample solution - http://aspnet.codeplex.com/SourceControl/changeset/view/bb167f0b0013#Samples/Net45/CS/WebApi/UploadXDocumentSample/ReadMe.txt

这篇关于WebAPI 请求流支持的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！