Azure Service Bus-避免并行处理来自同一设备的消息 [英] Azure Service Bus - avoid processing messages from same device in parallel

问题描述

许多设备正在发送消息，这些消息最终都在单个Azure Service Bus队列(或主题)中.我们希望并行处理多个消息，但我们希望避免在任何给定时间同时处理同一设备的两条消息.

A number of devices are sending messages which end up in a single Azure Service Bus queue (or topic). We want to process multiple messages in parallel, but we want to avoid concurrent processing of two messages of the same device at any given time.

以下图片说明了目标.有3个处理线程(实际上可能有几十个线程分布在多个服务器之间).每个框表示单个消息的处理时间，颜色显示其属于哪个设备.

The following pictures illustrates the goal. There are 3 processing threads (in reality there might be several dozens, distributed between several servers). Each box denotes processing time of a single message, and the color shows which device it belongs to.

您可以看到，在任何时候都不会出现来自同一设备的两条或更多条重叠消息.

You can see that at no point in time there are two or more overlapping messages originated from the same device.

由于涉及到多个处理服务器，我可以想象到，防止并发处理的唯一方法是使用设备ID作为分区键对消息进行分区，然后每个分区只有一个使用者:

As there are multiple processing servers involved, I can imagine that the only way to prevent concurrent processing is to partition messages with device ID as partitioning key, and then only have a single consumer for each partition:

因此，来自黄色设备"的所有消息都进入分区1，依此类推.

So, all messages from "yellow device" go to partition 1, and so on.

我仍然希望在单个进程中运行多个处理线程.现在，我们做一些简单的事情，例如

I still want to run multiple processing threads in the single process. Right now, we do something simple like

var client = QueueClient.CreateFromConnectionString(connectionString, queueName);
var options = new OnMessageOptions { MaxConcurrentCalls = x };
client.OnMessage(m =>
    {
        // Process...
        m.Complete();
    });

如何将并发限制合并到此类代码中?

How do I incorporate the concurrency limitation into such code?

我可以想象一些基于参与者或其他并发机制的客户端解决方案.但是有没有办法在经纪人级别解决这个问题?

I can imagine some client-side solutions based on actors or other concurrency mechanisms. But is there a way to solve that on Broker level?

推荐答案

这似乎是利用ASB s Sessions feature. You will be able to use OnMessage` API的不错选择，但是给定会话的处理将仅由单个使用者完成，而不是多个.另外，您将能够并发运行，并处理随之而来的负载.

This looks as a good candidate to take advantage of the ASBs Sessions feature. You will be able to useOnMessage` API, but processing of a given session will only be done by a single consumer, not multiple. Also, you'll be able to run concurrently, handling the load as it comes.

一个好的出发点是看此示例是您会发现的最好的doco.

A good starting point would be to look at QueueClient.AcceptMessageSessionAsync API. If you want a solid documentation with explanation how it works, this sample is the best doco you'll find.

这篇关于Azure Service Bus-避免并行处理来自同一设备的消息的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！