优化asyncronus HttpClient的请求 [英] Optimising asyncronus HttpClient requests

问题描述

我做了一个类来处理多个HTTP GET请求。它看起来是这样的：

I have made a class to handle multiple HTTP GET requests. It looks something like this:

public partial class MyHttpClass : IDisposable
{
    private HttpClient theClient;
    private string ApiBaseUrl = "https://example.com/";

    public MyHttpClass()
    {

        this.theClient = new HttpClient();
        this.theClient.BaseAddress = new Uri(ApiBaseUrl);
        this.theClient.DefaultRequestHeaders.Accept.Clear();
        this.theClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
    }

    public async Task<JObject> GetAsync(string reqUrl)
    {

        var returnObj = new JObject();
        var response = await this.theClient.GetAsync(reqUrl);

        if (response.IsSuccessStatusCode)
        {
            returnObj = await response.Content.ReadAsAsync<JObject>();
            Console.WriteLine("GET successful");

        }
        else
        {
            Console.WriteLine("GET failed");
        }

        return returnObj;
    }
    public void Dispose()
    {
        theClient.Dispose();
    }
}

我再排队多requets使用遍历Task.Run（），然后后Task.WaitAll（）的方式：

I am then queueing multiple requets by using a loop over Task.Run() and then after Task.WaitAll() in the manner of:

public async Task Start()
{
    foreach(var item in list)
    {
        taskList.Add(Task.Run(() => this.GetThing(item)));
    }
    Task.WaitAll(taskList.ToArray());
}

public async Task GetThing(string url)
{
    var response = await this.theClient.GetAsync(url);

    // some code to process and save response

}

据definitiely作品比synchonus运行速度更快，但它并不像快如我所料。根据其他方面的建议，我认为当地的线程池正在放缓我失望。 MSDN建议我应该将其指定为长时间运行的任务，但我看不到的方式来做到这一点调用它是这样的。

It definitiely works faster than synchonus operation but it is not as fast as I expected. Based on other advice I think the local threadpool is slowing me down. MSDN suggest I should specify it as a long running task but I can't see a way to do that calling it like this.

现在我还没有得到进入限制线程，我只是做批次，检测速度快，发现正确的方法。

Right now I haven't got into limiting threads, I am just doing batches and testing speed to discover the right approach.

任何人都可以提出一些领域，我看提高速度？

Can anyone suggest some areas for me to look at to increase the speed?

推荐答案

所以，你已经设置后，你的<一个href="https://msdn.microsoft.com/en-us/library/system.net.servicepointmanager.defaultconnectionlimit%28v=vs.110%29.aspx"相对=nofollow> DefaultConnectionLimit 以一个漂亮的高一些，或者只是设法你打的主机连接的ServicePoint的ConnectionLimit：

So, after you've set your DefaultConnectionLimit to a nice high number, or just the ConnectionLimit of the ServicePoint that manages connections to the host you are hitting:

ServicePointManager
    .FindServicePoint(new Uri("https://example.com/"))
    .ConnectionLimit = 1000;

的code的唯一嫌疑人位就是你开始一切......

the only suspect bit of code is where you start everything...

public async Task Start()
{
    foreach(var item in list)
    {
        taskList.Add(Task.Run(() => this.GetThing(item)));
    }
    Task.WaitAll(taskList.ToArray());
}

此可以减少到

var tasks = list.Select(this.GetThing);

要创建任务（你的异步方法返回热（运行）的任务......无需重复包装与 Task.Run ）

to create the tasks (your async methods return hot (running) tasks... no need to double wrap with Task.Run)

然后，而阻断等待他们完成，等待异步，而不是：

Then, rather that blocking while waiting for them to complete, wait asynchronously instead:

await Task.WhenAll(tasks);

这篇关于优化asyncronus HttpClient的请求的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！