如何并行处理MSMQ消息 [英] How to process MSMQ messages in parallel

问题描述

我正在写一个窗口服务消费MSMQ消息。这项服务将有高活性的时期（进来的速度非常快80K消息）和长期不活动的（可能是数天没有新的消息）。

I'm writing a windows service to consume MSMQ messages. The service will have periods of high activity (80k messages coming in very quickly) and long periods of inactivity (could be several days without a new message).

处理的信息是非常网络结合，让我获得了很大的好处了并行性。但是，在非活动期间，我不想占用了一堆等待消息未马上就要随时线程。

Processing the messages is very network-bound, so I get a big benefit out of parallelism. But during periods of inactivity, I don't want to tie up a bunch of threads waiting for messages that aren't coming anytime soon.

的MSMQ接口显得非常专注于一个同步的工作流程 - 得到一个消息，对它进行处理，得到另一个，等等。我应该如何组织我的code，这样我可以在高活动期间利用并行但在不活动期间不打领带了一堆线程？奖励积分的使用TPL。伪code将AP preciated。

The MSMQ interface seems to be very focused on a synchronous workflow - get one message, process it, get another, etc. How should I structure my code so that I can take advantage of parallelism during periods of high activity but not tie up a bunch of threads during periods of no activity? Bonus points for using the TPL. Pseudocode would be appreciated.

推荐答案

我已经做了配发MSMQ（包括移动实现）多年来，你在的特性正确的同步工作流程。这并不是说你不能把各种信息包围，并通过第三方物流在不同的内核可以处理他们...的限制因素正在读取/写入队列......本质上是一个串行操作。在一次（电脑与8核），比如你不能发送8的消息。

I have done allot of MSMQ (including mobile implementations) over the years and you are correct in the characterization of "synchronous workflow." It's not that you can't take the various message envelops and process them across the different cores via TPL ... the limiting factor is reading / writing to the queue ... inherently a serial operation. You can't send 8 messages at once (a computer with 8 cores) for example.

我也有类似的需求（不使用System.Messaging命名空间），从一本书一些帮助坎贝尔和约翰逊解决它，我读并行编程与Microsoft.NET。

I had a similar need (without using the System.Messaging namespace) and solved it with some help from a book I read "Parallel Programming with Microsoft.NET" by Campbell and Johnson.

看看他们的并行任务一章，并使用与每个线程的本地队列的工作处理合作的全球队列专门的部分（即TPL）使用工作窃取算法来执行负载平衡。我模仿我的解决方案，在某种程度上，他们的榜样之后。我的系统的最终版本曾在其业绩的巨大差异（每秒23消息，200）。

Check out their "parallel tasks" chapter and specifically the part of using a global queue that cooperates with per-thread local queues for work processing (i.e., the TPL) that use a "work stealing" algorithm to perform load balancing. I modeled my solution, in part, after their example. The final version of my system had a huge difference in its performance (from 23 messages per second to over 200).

根据需要多长时间你的系统从0到去八万人，你要采取跨多个服务器相同的设计和s $ P $填充它（每个具有多个处理器和多个内核）。从理论上讲我的设置将需要超过700分钟的时间一点点地干掉所有80K，因此加入了第二个计算机，将削减下来大约〜3分20秒时，等，等，等诀窍是工作窃取逻辑。

Depending on how long it takes your system to go from 0 to the 80,000, you’ll want to take the same design and spread it across multiple servers (each with multiple processors and multiple cores). In theory my setup would require a little less than 7 minute to polish off all 80K, so by adding a 2nd computer it would cut that down to about ~3 minutes and 20 seconds, etc., etc., etc. The trick is the work stealing logic.

深思......

一个快速编辑：BTW电脑是戴尔T7500工作站采用双四核Xeon处理器@ 3GHz的，24 GB的RAM，Windows 7旗舰版64位版本

A quick edit: BTW the computer is a Dell T7500 workstation with dual quad core Xeons @ 3GHz, 24 GB of RAM, Windows 7 Ultimate 64-bit edition.

这篇关于如何并行处理MSMQ消息的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！