在实现生产者/消费者模式时使用Task.Yield克服ThreadPool饥饿 [英] Using Task.Yield to overcome ThreadPool starvation while implementing producer/consumer pattern

问题描述

回答问题: Task.Yield-实际用法? 我建议使用Task.Yield允许池线程由其他任务重用.在这种模式下:

Answering the question: Task.Yield - real usages? I proposed to use Task.Yield allowing a pool thread to be reused by other tasks. In such pattern:

  CancellationTokenSource cts;
  void Start()
  {
        cts = new CancellationTokenSource();

        // run async operation
        var task = Task.Run(() => SomeWork(cts.Token), cts.Token);
        // wait for completion
        // after the completion handle the result/ cancellation/ errors
    }

    async Task<int> SomeWork(CancellationToken cancellationToken)
    {
        int result = 0;

        bool loopAgain = true;
        while (loopAgain)
        {
            // do something ... means a substantial work or a micro batch here - not processing a single byte

            loopAgain = /* check for loop end && */  cancellationToken.IsCancellationRequested;
            if (loopAgain) {
                // reschedule  the task to the threadpool and free this thread for other waiting tasks
                await Task.Yield();
            }
        }
        cancellationToken.ThrowIfCancellationRequested();
        return result;
    }

    void Cancel()
    {
        // request cancelation
        cts.Cancel();
    }

但是一位用户写了

我不认为使用Task.Yield来克服ThreadPool饥饿 实施生产者/消费者模式是一个好主意.我建议你 问一个单独的问题，是否要详细说明原因.

I don't think using Task.Yield to overcome ThreadPool starvation while implementing producer/consumer pattern is a good idea. I suggest you ask a separate question if you want to go into details as to why.

任何人都知道，为什么不是一个好主意?

Anybody knows, why is not a good idea?

推荐答案

在您的问题的注释中还有一些要点.作为您引用的用户，我想总结一下:使用正确的工具完成工作.

There are some good points left in the comments to your question. Being the user you quoted, I'd just like to sum it up: use the right tool for the job.

使用ThreadPool感觉不像是执行多个连续的CPU绑定任务的正确工具，即使您尝试通过将协作转化为状态机来组织一些协作执行，这些状态机也会与await Task.Yield()相互产生CPU时间.线程切换是相当昂贵的.通过在紧密循环上执行await Task.Yield()，会增加大量开销.此外，您永远不要接管整个ThreadPool，因为.NET框架(和底层OS进程)可能需要它来做其他事情.与此相关的是，TPL甚至具有TaskCreationOptions.LongRunning选项，该选项请求不要在ThreadPool线程上运行该任务(相反，它会创建一个普通线程，并且在后台隐藏new Thread().)

Using ThreadPool doesn't feel like the right tool for executing multiple continuous CPU-bound tasks, even if you try to organize some cooperative execution by turning them into state machines which yield CPU time to each other with await Task.Yield(). Thread switching is rather expensive; by doing await Task.Yield() on a tight loop you add a significant overhead. Besides, you should never take over the whole ThreadPool, as the .NET framework (and the underlying OS process) may need it for other things. On a related note, TPL even has the TaskCreationOptions.LongRunning option that requests to not run the task on a ThreadPool thread (rather, it creates a normal thread with new Thread() behind the scene).

也就是说，在某些专用的，池外线程上使用有限并行性的 custom TaskScheduler可能与单个长期运行的任务具有线程亲和力 另一回事.至少，await延续将发布在同一线程上，这将有助于减少切换开销.这使我想起了我之前尝试使用 ThreadAffinityTaskScheduler 来解决的另一个问题.

That said, using a custom TaskScheduler with limited parallelism on some dedicated, out-of-pool threads with thread affinity for individual long-running tasks might be a different thing. At least, await continuations would be posted on the same thread, which should help reducing the switching overhead. This reminds me of a different problem I was trying to solve a while ago with ThreadAffinityTaskScheduler.

尽管如此，根据特定情况，通常最好使用现有的成熟工具和经过测试的工具.仅举几例:平行类， TPL数据流，反应式扩展.

Still, depending on a particular scenario, it's usually better to use an existing well-established and tested tool. To name a few: Parallel Class, TPL Dataflow, System.Threading.Channels, Reactive Extensions.

还有各种现有的行业实力解决方案可以处理发布-订阅模式(RabbitMQ，PubNub，Redis，Azure服务总线，Firebase云消息传递(FCM)，Amazon Simple Queue Service(SQS)等).

There is also a whole range of existing industrial-strength solutions to deal with Publish-Subscribe pattern (RabbitMQ, PubNub, Redis, Azure Service Bus, Firebase Cloud Messaging (FCM), Amazon Simple Queue Service (SQS) etc).

这篇关于在实现生产者/消费者模式时使用Task.Yield克服ThreadPool饥饿的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！