JMS 拓扑(具有多个消费者的队列)和消息组 [英] JMS topology (Queue with multiple consumers) and message groups

问题描述

这是我们设置的简化/示意图拓扑

here is a simplified / schematic topology we set up

   [Front server 1 (message producer)]  [Front server n (message producer)]
                   |                                    |        
                   |________________    ________________| 
                                    |  |         
                         [Messaging Server (HornetQ)]
                                    |  |
                    ________________|  |________________
                    |                                   |
   [Task server 1 (Message Driven Bean)]          [Task server n (MDB)]  

• 每个节点(服务器)都是一个独立的(无集群)jboss 应用服务器 (Jboss-as7)，包括消息传递服务器.
• 消息传递服务器部署了许多 JMS 队列.
• 每个任务服务器为每个队列部署一个 MDB，其中包含许多消费者.
• 所有消息生产者都使用相同的入站适配器，所有消息消费者 (MDB) 都使用相同的出站适配器.事实上，所有前端节点都完全相同(相同的 AS、相同的配置、相同的部署工件)，所有服务器节点都相同.

现在这是我的问题:

应用是多租户的，对于给定的队列，对于给定的租户，某些任务(消息处理)一定不能并行处理，我们这样设置消息分组来处理这个限制.消息组是租户名称，由消息生产者在发送消息时设置:

the application is a multi-tenant one, for a given queue, some tasks (message processing) must not be processed in parallel for a given tenant, we set up so message grouping to handle this constraint. The message group is the tenant name and is set by the message producer when sending the message :

message.setStringProperty("JMSXGroupID", tenantName);

在一个平台上，我们有 1000 多个租户(因此有 1000 多个不同的消息组)，并且对于给定的队列，每个服务器有 3 个消费者和 3 个任务服务器.

On a platform we have 1000+ tenant (so 1000+ different message groups) and for a given queue 3 consumers per server and 3 tasks servers.

在消息服务器上监控此队列时，我们可以看到队列中有数千条消息和 9 个消费者.我们预计消息会按 9 个 9 发送，但实际上发送中的消息计数永远不会超过 1.

When monitoring this queue on the messaging server we can so see thousands messages in queue and 9 consumers. We expects message to be delivered 9 by 9 but in practice the in-delivery message count never exceed 1.

这里有什么问题?拓扑是否适合我们的需求?

What's the problem here ? is the topology suitable for our needs ?

推荐答案

来自 jboss hornetq 支持论坛的回答(来源 Justin Bertram):

Answer from jboss hornetq support forum (crédits Justin Bertram):

我确定您知道，队列具有先进先出(即 FIFO)语义.分组消息基本上就像一个瓶颈队列，因为它们确保串行消息处理.这是一个简单的例如...

As I'm sure you're aware, a queue has first-in-first-out (i.e. FIFO) semantics. Grouped messages basically act like a bottleneck on the queue since they ensure serial message processing. Here's a simple example...

考虑包含两个消息的组 A、B 和 C每个队列中总共有 6 条消息.还认为他们的顺序是 A、A、B、B、C、C.现在考虑 3 个不同的消费者每个消费不同的群体.A组的消费者将接收第一条消息，处理它并确认它，以便它从队列中删除.然后它会收到第二条消息，处理它，并确认它以便从队列中删除它.在 A 组的消费者忙于处理这 2 条消息期间不能从队列中消费其他消息.只有当第二条消息被确认，B 组的消费者实际上可以接收 B 组的第一条消息.一旦 B 组的消费者确认它的两条消息，C 组的消费者最终可以在其组中接收消息.这种行为受队列的 FIFO 语义.B组的消息不能越级A 组中的消息，并在所有消息之前被消费A组被消耗.C组中的消息也是如此.

Consider groups A, B, and C which contain two messages each for a total of 6 messages in the queue. Also consider that they are in the order A, A, B, B, C, C. Now consider 3 different consumers each consuming a different group. The consumer for group A will receive the first message, process it, and acknowledge it so that it is removed from the queue. Then it will receive the second message, process it, and acknowledge it so that it is removed from the queue. During the time the consumer for group A is busy with these 2 messages no other messages can be consumed from the queue. Only when the second message is acknowledged can the consumer for group B actually receive the first message in group B. Once the consumer for group B acknowledges both of its messages the consumer for group C can finally receive the messages in its group. This behavior is governed by the FIFO semantics of the queue. The messages in group B can't leap-frog the messages in group A and be consumed before all the messages in group A are consumed. The same goes for the messages in group C.

既然你所有的消息都在一个组里，我从没想过在交付计数超过 1.

Since all your messages are in a group I would never expect the in-delivery count to exceed 1.

这篇关于JMS 拓扑(具有多个消费者的队列)和消息组的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！