在ASP.NET Core中使用HttpClient或WebRequest更好地异步异步地逐行读取文件的内容吗? [英] Is it better to use HttpClient or WebRequest in ASP.NET Core to read the content of a file line by line asynchronously remotely?
问题描述
我计划使用 https://github.com/Dasync/AsyncEnumerable逐行异步读取远程文件(因为还没有异步流[可能是C#8]:
I plan to read a remote file line by line asynchronously using https://github.com/Dasync/AsyncEnumerable (since there is not yet Async Streams [C# 8 maybe]: https://github.com/dotnet/csharplang/blob/master/proposals/async-streams.md):
public static class StringExtensions
{
public static AsyncEnumerable<string> ReadLinesAsyncViaHttpClient(this string uri)
{
return new AsyncEnumerable<string>(async yield =>
{
using (var httpClient = new HttpClient())
{
using (var responseStream = await httpClient.GetStreamAsync(uri))
{
using (var streamReader = new StreamReader(responseStream))
{
while(true)
{
var line = await streamReader.ReadLineAsync();
if (line != null)
{
await yield.ReturnAsync(line);
}
else
{
return;
}
}
}
}
}
});
}
public static AsyncEnumerable<string> ReadLinesAsyncViaWebRequest(this string uri)
{
return new AsyncEnumerable<string>(async yield =>
{
var request = WebRequest.Create(uri);
using (var response = request.GetResponse())
{
using (var responseStream = response.GetResponseStream())
{
using (var streamReader = new StreamReader(responseStream))
{
while(true)
{
var line = await streamReader.ReadLineAsync();
if (line != null)
{
await yield.ReturnAsync(line);
}
else
{
return;
}
}
}
}
}
});
}
}
似乎它们都可以在一个简单的控制台应用程序中正常运行，如下所示:
It seems that they both run just fine in a simple Console application like below:
public class Program
{
public static async Task Main(string[] args)
{
// Or any other remote file
const string url = @"https://gist.githubusercontent.com/dgrtwo/a30d99baa9b7bfc9f2440b355ddd1f75/raw/700ab5bb0b5f8f5a14377f5103dbe921d4238216/by_tag_year.csv";
await url.ReadLinesAsyncViaWebRequest().ForEachAsync(line =>
{
Console.WriteLine(line, Color.GreenYellow);
});
await url.ReadLinesAsyncViaHttpClient().ForEachAsync(line =>
{
Console.WriteLine(line, Color.Purple);
});
}
}
...但是如果将它用作ASP.NET Core WebAPI的一部分来处理行，然后使用PushStreamContent推送行，我会有些担心:
... but I have some concerns if it is used as part of an ASP.NET Core WebAPI to process the lines and then push them using PushStreamContent:
- https://docs.microsoft.com/en-us/previous-versions/aspnet/hh995285(v=vs.108)
- https://blog.stephencleary.com/2016/10/async-pushstreamcontent.html
想法是要建立一个利用 async / await 的数据管道，以使使用的线程数尽可能少，并且避免增加内存(利用了AsyncEnumerable的类似枚举的功能).
The idea would be to have a pipeline of data which leverages async / await so that the number of threads in use is as low as possible and also to avoid an increase in memory (which leverage the enumerable-like feature of AsyncEnumerable).
我阅读了几篇文章，但似乎都是非.NET Core版本，而且我真的不知道是否有一些潜在的性能问题/关于我想要实现的警告?
I read several articles but it seems it's all non .NET Core versions and I don't really know if there would be some potential performance issues / caveats in regard to what I would like to achieve?
- HttpRequest，HttpWebRequest和WebRequest之间的区别
- http://www.diogonunes.com/blog/webclient-vs-httpclient-vs-httpwebrequest/
业务"案例的一个例子是:
An example of "business" case would be:
using System;
using System.Collections.Async;
using System.IO;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Mvc;
namespace WebApplicationTest.Controllers
{
[Route("api/[controller]")]
[ApiController]
public class DumbValuesController : ControllerBase
{
private static readonly Random Random = new Random();
// GET api/values
[HttpGet]
public async Task<IActionResult> DumbGetAsync([FromQuery] string fileUri)
{
using (var streamWriter = new StreamWriter(HttpContext.Response.Body))
{
await fileUri.ReadLinesAsyncViaHttpClient().ForEachAsync(async line =>
{
// Some dumb process on each (maybe big line)
line += Random.Next(0, 100 + 1);
await streamWriter.WriteLineAsync(line);
});
}
return Ok();
}
}
}
推荐答案
我们可以访问.NET Core的源代码.这样你就可以看.
We have access to the source code for .NET Core. So you can look.
两者的基础实现最终都使用 HttpClientHandler (该类的实现分为4个文件).
The underlying implementation of both end up using HttpClientHandler (the implementation of that class is split up into 4 files).
You can see this from the source code of both HttpClient and HttpWebRequest (which WebRequest uses).
所以我怀疑您不会注意到两者的性能差异.
So I suspect you won't notice any difference in the performance of either.
HttpClient 是要编写的最新版本，因此鼓励使用它.并且由于您在文章中提到的原因，您链接到: http://www.diogonunes.com/blog/webclient-vs-httpclient-vs-httpwebrequest/
HttpClient is the latest one to be written, so that's why its use is encouraged. And for the reasons mentioned in the article you linked to: http://www.diogonunes.com/blog/webclient-vs-httpclient-vs-httpwebrequest/
这篇关于在ASP.NET Core中使用HttpClient或WebRequest更好地异步异步地逐行读取文件的内容吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
