扩展MSMQ并保证交付顺序 [英] Scaling MSMQ and guaranteed order of delivery

问题描述

Hi All

我需要处理来自几千台设备的大量传入消息（每小时约250000）。在写入数据库之前，需要对消息进行一些处理。对于设备，消息必须按发送顺序写入数据库。

我正在考虑使用MSMQ接收和保存消息，然后让多个消息处理器从队列中读取，进行处理并写入数据库。消息处理器的数量将根据可扩展性进行调整。

我的问题是尝试将多个消息处理器与每个设备的保证交付顺序组合在一起。如果设备在队列上放置两条消息，并且两个不同的消息处理器分别读取一条消息，则无法保证交付顺序。

我有一个潜在的解决方案，即确保所有消息来自设备仅通过一个特定的消息处理器进行路由。因此消息处理器1处理来自设备1-50的消息，处理器2处理来自设备51-100等的消息。为此，我将不得不编写我自己的路由器

有没有更好的方法来实现这一点？ MSMQ本身可以帮助吗？

亲切的问候

Hi All

I need to process a large number of incoming messages (about 250000 per hour) coming from a couple of thousand devices. Some processing needs to occur on the messages before being written to a database. For a device, the messages MUST be written to the database in order of sending.

I am thinking of using MSMQ to receive and hold the messages, and then have multiple message processors reading from the queue, doing the processing and writing to the database. The number of message processors would be tuned for scalability.

My problem is in trying to combine multiple message processors with guaranteed order of delivery per device. If a device puts two messages on the queue, and two different message processors read one each, there is no guaranteed order of delivery.

I have a potential solution in mind, which is to ensure that all messages from a device are only routed through one particular message processor. So message processor 1 deals with messages from devices 1-50, processor 2 deals with messages from devices 51-100 etc. To do this I would have to write my own router

Is there a better way to achieve this? Can MSMQ itself help?

Kind regards

推荐答案

你好Andy，

可能有两种方式这里：
1]同步：
这里它非常直接，所有内容都是从输入到消息队列的序列化，直到数据库I / O.这将是最好的选择。因为可扩展性是您正在寻找的，所以您需要查找以下选项。

2]异步
Msmq肯定会按照您可以直接选择的条目顺序存储消息。开始处理。但是，因为你提到了从几个设备接收数据包{假设设备=网络上的计算机}同时。在设备发送消息之前使用某种包含序列和唯一设备ID的标头修复消息结构的格式肯定会为您放松很多事情，并且您可以使用单个消息处理器或基于多消息处理器在你的设计和因此要求保持你需要将它们写入db的顺序。 Afaik MSMQ并没有直接在这里给你你想要的东西。

希望这有帮助。

Hi Andy,

     There might be 2 ways here:
1] Synchronous:
     Here it is pretty much straight forward and everything is serialized right from the input to your message queue till the database I/O. And this would be the best option. Bust since scalability is what you are looking for you need to lookup the following option.

2]  Asynchronous
    Msmq will definitely store the messages in the order of entry which you can pick up directly & start processing. But since you are mentioning about receiving packets from several devices{presuming device = computers over the network for example} simultaneously. Fixing a format for the message structure with some sort of a header containing the sequence and a unique device ID before the devices send out the message would definitely easen a lot of things for you and you can have a single message processor or multi message processor based on your design & requirement thus maintaining the order in which u need to write them to the db. Afaik MSMQ doesn't directly give you what you are looking for over here.

Hope this helps.

这篇关于扩展MSMQ并保证交付顺序的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！