请求在Azure AppService中排队，尽管它在线程池中有足够的线程 [英] Requests are queuing in Azure AppService though it has enough threads in threadpool

问题描述

我使用asp.net webapi编写了一个api，并将其部署在azure中作为Appservice。我的控制器的名称是TestController，我的操作方法类似于贝娄。

I have written an api using asp.net webapi and deployed it in azure as Appservice. Name of my controller is TestController and My action method is something like bellow.

    [Route("Test/Get")]
    public string Get()
    {
        Thread.Sleep(10000);
        return "value";
    }

因此，对于每个请求，它应该等待10秒，然后返回字符串value 。我还编写了另一个端点来查看threadpool中用于执行请求的线程数。这个动作类似于贝娄。

So for each request it should wait for 10 sec before return string "value". I have also written another endpoint to see the number of threads in threadpool working for executing requests. That action is something like bellow.

    [Route("Test/ThreadInfo")]
    public ThreadPoolInfo Get()
    {
        int availableWorker, availableIO;
        int maxWorker, maxIO;

        ThreadPool.GetAvailableThreads(out availableWorker, out availableIO);
        ThreadPool.GetMaxThreads(out maxWorker, out maxIO);

        return new ThreadPoolInfo
        {
            AvailableWorkerThreads = availableWorker,
            MaxWorkerThreads = maxWorker,
            OccupiedThreads = maxWorker - availableWorker
        };
    }

现在，当我们同时向Test / Get端点拨打29个电话时，它几乎需要11秒即可获得所有请求。因此服务器在11个线程中同时执行所有请求。要查看线程状态，在调用Test / Get后立即调用Test / ThreadInfo会立即返回（无需等待）
{
AvailableWorkerThreads：8161，
MaxWorkerThreads：8191 ，
OccupiedThreads：30
}

Now when We make 29 get calls concurrently to Test/Get endpoint it takes almost 11 seconds to get succeed all requests. So server executes all the requests concurrently in 11 threads. To see the threads status, making call to Test/ThreadInfo right after making call to Test/Get returns immediately(without waiting) { "AvailableWorkerThreads": 8161, "MaxWorkerThreads": 8191, "OccupiedThreads": 30 }

似乎29个线程正在执行Test / Get请求，1个线程正在执行Test / ThreadInfo请求。

Seems 29 threads are executing Test/Get requests and 1 thread is executing Test/ThreadInfo request.

当我接到60次调用Test / Get时，需要将近36秒才能获得成功。调用Test / ThreadInfo（需要一些时间）返回
{
AvailableWorkerThreads：8161，
MaxWorkerThreads：8191，
OccupiedThreads：30
}

When I make 60 get calls to Test/Get it takes almost 36 seconds to get succeed. Making call to Test/ThreadInfo(takes some time) returns { "AvailableWorkerThreads": 8161, "MaxWorkerThreads": 8191, "OccupiedThreads": 30 }

如果我们增加请求数量，则OccupiedThreads的值会增加。与1000次请求一样，它需要2分22秒，且OccupiedThreads的值为129.

If we increase requests number, value of OccupiedThreads increases. Like for 1000 requests it takes 2 min 22 sec and value of OccupiedThreads is 129.

看似请求并在30次并发呼叫后排队，尽管在WorkerThread中有很多线程可用。逐渐增加并发执行的线程，但这还不够（1000请求为129）。

Seems request and getting queued after 30 concurrent call though lot of threads are available in WorkerThread. Gradually it increases thread for concurrent execution but that is not enough(129 for 1000 request).

由于我们的服务有很多IO调用（其中一些是外部api调用，一些是数据库查询），因此延迟也很高。由于我们使用所有IO调用异步方式，因此服务器可以同时处理大量请求，但是当处理器正在进行实际工作时我们需要更多的并发性。我们正在使用S2服务计划和一个实例。增加实例会增加并发性，但我们需要单个实例的更多并发。

As our services has lot of IO call(some of them are external api call and some are database query) the latency is also high. As we are using all IO calls async way so server can serve lot of request concurrently but we need more concurrency when processor are doing real work. We are using S2 service plan with one instance. Increasing instance will increase concurrency but we need more concurrency from single instance.

在阅读IIS上的一些博客和文档后，我们看到有一个设置 minFreeThreads 。如果线程池中的可用线程数低于此设置的值，则IIS开始对请求进行排队。 appservice有这样的东西吗？是否真的有可能从azure app服务获得更多的并发性，或者我们在那里缺少一些配置？

After reading some blog and documentation on IIS we have seen there is a setting minFreeThreads. If the number of available threads in the thread pool falls bellow the value for this setting IIS starts to queue request. Is there anything in appservice like this? And Is it really possible to get more concurrency from azure app service or we are missing some configuration there?

推荐答案

最后得到了回答我的问题。问题是ASP.NET线程池维护了一个线程池，这些线程已经引起了线程初始化成本并且易于重用。 .NET线程池也是自我调整的。它监视CPU和其他资源利用率，并根据需要添加新线程或修剪线程池大小。当有大量请求且池中没有足够的线程可用时，线程池开始在池中添加新线程，在此之前它运行自己的算法以查看内存状态和CPU使用系统需要长时间的时间，这就是为什么我们看到池中的工作线程缓慢增加并且大量请求排队。但幸运的是，在线程池切换到添加新线程的算法之前，可以选择设置工作线程的数量。代码类似于贝娄。

At last got the answer for my question. The thing is that The ASP.NET thread pool maintains a pool of threads that have already incurred the thread initialization costs and are easy to reuse. The .NET thread pool is also self-tuning. It monitors CPU and other resource utilization, and it adds new threads or trims the thread pool size as needed. When there are lot of requests and not enough thread in pool is available then thread pool starts to add new threads in the pool and before that it runs its own algorithm to see the status of memory and cpu use of the system which takes long amount of time and that is why we see slowly increase of worker thread in the pool and get lot of request queued. But luckly there is an option to set the number of worker thread before thread pool switches to an algorithm to add new thread. The code is something like bellow.

    public string Settings()
    {
        int minWorker, minIOC;
        ThreadPool.GetMinThreads(out minWorker, out minIOC);

        if (ThreadPool.SetMinThreads(300, minIOC)){ 
            return "The minimum number of threads was set successfully.";
        }
        else
        {
            return "The minimum number of threads was not changed.";
        }

    }

这里 ThreadPool.SetMinThreads （300，minIOC）设置线程池在切换到添加或删除线程的算法之前将创建的最小线程的值。我已将此方法添加为我的webapi控制器的操作，然后在我向Test / Get端点发出300个并发请求时发出请求后运行此操作，所有请求都在11秒内运行并且没有请求排队。

Here ThreadPool.SetMinThreads(300, minIOC) is setting the value of minimum threads threadpool will create before switching to an algorithm for adding or removing thread. I have Added this method as an action of my webapi controller and then after running this action by making a request when I made 300 concurrent request to Test/Get endpoint all was running and completed in 11 seconds and no request were queued.

这篇关于请求在Azure AppService中排队，尽管它在线程池中有足够的线程的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！