用有限的资源并行化工作 [英] Parallelizing jobs with limited resources

问题描述

亲爱的所有



我正在尝试并行化部分应用程序，但我遇到了一些问题......



我需要计算N个工作。一个作业的资源很大，以至于无法准备所有N个作业，并让它们在几个线程之后进行处理。因此，我需要用M个项目提供有限的工作队列（其中M比N小很多）



所以我需要管理的是：

1.）有N个工作要执行

2.）有一个最大M项的作业队列，由生产者线程提供（？）
3.）有一个线程池，其中有K个工作线程等待工作



如何管理/同步所有这些？



- 谁将分配/调用工作线程的工作？

- 工作者应该在完成工作后 - 检查工作为新工作排队？或者工作线程是否只是转到等待列表而另一个线程将待处理的工作分派给工作线程？



有人知道这种模式吗？


非常感谢您提前。

问候，idle63

Dear all

I’m trying to parallelize a part of our application but I have some problems….

I need to calculate N jobs. Resources for one job are that big that it is not possible to prepare all N jobs and let them afterwards process by several threads. Therefore I need to feed a limited job queue with M items (where M is pretty much smaller than N)

So what I need to manage is:
1.) There are N Jobs to execute
2.) There is a job queue with maximal M items, feed by a producer thread (?)
3.) There is a thread pool with K worker threads waiting for work

How can I manage/synchronize all these?

- Who will assign/invoke a worker thread its work?
- Should a worker thread -after finishing its job- check the job queue by itself for new work? Or should the worker thread simply go to a "waiting list" and another thread dispatches pending work to worker threads?

Does somebody know a pattern for this?

Thank you very much in advance.
Regards, idle63

推荐答案

查看生产者 - 消费者数据流模式：

http://msdn.microsoft.com/en-us/library/hh228601.aspx [<一个href =http://msdn.microsoft.com/en-us/library/hh228601.aspxtarget =_ blanktitle =新窗口> ^ ]



祝你好运！

Have a look at a the Producer-Consumer Dataflow Pattern:
http://msdn.microsoft.com/en-us/library/hh228601.aspx[^]

Good luck!

这篇关于用有限的资源并行化工作的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！