在C＃中处理项目的队列异步 [英] Processing a queue of items asynchronously in C#

问题描述

我试图创建一个处理工作队列系统。该系统具有以下规格：

1. 该系统由两部分组成，工作分配器和工人。


2. 有在同一时间运行工人的人数一套上限。该上限是大于一。


3. 要避免具有相同的任务的问题正在处理两次，仅存在单一的工作分配器。





你会用什么设计，建立这样一个系统？下面是我在想什么：






1. 创建队列集合，一个队列每个工人


2. 创建工作分配器一个计时器。它的工作将是填充队列。


3. 创建每个工人一个计时器，在一个队列对象传递作为对象的状态来表示其工作量


4. 删除，并添加到队列中，而他们被锁定。


5. 使用的递增，而锁定的递减计数器，以确保不超过职工工作的规定数以上是在同一时间运行。






我觉得必须有一个更好的方式来做到这一点。你有什么建议？我应该从定时器切换到线程工人？如果线程只是旋转/等待而队列为空？如果线程关闭，并已工作分配器有条件创造一个新的？




解决方案

我不知道你的任务多长时间运行，但它似乎是最好的办法将是使用线程池。
此外，我会用了，居然都用过，只有一个中央队列 - 仅此便可去除一些复杂性。
我有一个线程处理该队列，并在这日将是队列的任务在你的情况下，项目的动作。





作为使队列线程安全的，有为了这个目的（ MSDN ，<在System.Collections.Concurrent一个ConcurrentQueue A HREF =http://geekswithblogs.net/BlackRabbitCoder/archive/2010/06/07/c-system.collections.concurrent.concurrentqueue-vs.-queue.aspx>基准VS锁定队列的）。





现在，扔在一个BlockingCollection（ MSDN ），你有你所需要的

  BlockingCollection<数据分组> sendQueue =新BlockingCollection<数据分组>（新ConcurrentQueue<数据分组>（））; 
，而（真）
 {
 VAR包= sendQueue.Take（）; //此块如果在队列中没有的项目。 
 ThreadPool.QueueUserWorkItem（州=> 
 {
 VAR数据=（Packet）的状态; 
 //做你必须做
}，包）; 
} 
  

和地方存在着 sendQueue.Add东西（包）;





综上所述，






1. 所有打工皇帝


2. 一个线程从队列中
   出队，并把它传递给线程池。一个队列


$ ; b
$ b

我觉得这是它





PS：如果你要控制线程的数量，请使用智能线程池的建议通过josh3736




I am trying to create a system that processes a queue of work. The system has the following specifications:

1. The system has two components, a work assigner and a worker.
2. There is a set upper limit on the number of workers running at the same time. That upper limit is greater than one.
3. To avoid problems with the same task being worked on twice, there is only a single work assigner.

What design would you use to create such a system? Here is what I am thinking:

1. Create a collection of queues, one queue for each worker
2. Create a Timer for the work assigner. Its job would be to populate the queues.
3. Create a Timer for each worker, passing in a queue object as the object state to represent its workload
4. Remove and add to the queues while they are locked.
5. Use a counter that is incremented and decremented while locked to ensure that no more than the specified number of worker tasks is running at the same time.

I feel like there must be a better way to do this. What would you recommend? Should I switch from Timers to Threads for the workers? Should the Threads just spin/wait while the queue is empty? Should the threads close and have the work assigner conditionally create a new one?

解决方案

I don't know how long your tasks will be running, but it seems that the best thing to do would be to use ThreadPool. Furthermore, I would use, and actually have used, only one central queue - that alone will remove some complexity. I have one Thread that handles the queue and does an action on the item in your case it would be to queue a task.

As for making the queue threadsafe, there is a ConcurrentQueue in System.Collections.Concurrent for that very purpose (msdn, benchmark vs locking queue).

Now, throw in a BlockingCollection (msdn) and you have all you need.

        BlockingCollection<Packet> sendQueue = new BlockingCollection<Packet>(new ConcurrentQueue<Packet>());
        while (true)
        {
            var packet = sendQueue.Take(); //this blocks if there are no items in the queue.
            ThreadPool.QueueUserWorkItem(state =>
            {
               var data = (Packet)state;
               //do whatever you have to do
            }, packet );
        }

and somewhere there is something that sendQueue.Add(packet);

To sum up,

1. One queue for all "workers"
2. One thread that dequeues from the queue and passes it to the ThreadPool.

I think that's it.

ps: if you have to control the amount of threads, use "Smart Thread Pool" as suggested by josh3736

这篇关于在C＃中处理项目的队列异步的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！