如何在保证每个实体的FIFO的同时处理消息？ [英] How do you process messages in parallel while ensuring FIFO per entity?
问题描述
假设您在系统中有一个实体，例如Person，并且您希望处理修改各种Person实体的事件。重要的是：
Let's say you have an entity, say, "Person" in your system and you want to process events that modify various Person entities. It is important that:
- 同一人的事件按FIFO顺序处理
- 事件流由不同的线程/进程并行处理
我们有一个实现使用共享数据库和锁来解决这个问题。线程竞争获取一个Person的锁，然后在获取锁后按顺序处理事件。我们想移动到一个消息队列，以避免轮询和锁定，我们认为这将减少DB上的负载，并简化消费者代码的实现。
We have an implementation that solves this using a shared database and locks. Threads compete to acquire the lock for a Person and then process events in order after acquiring the lock. We'd like to move to a message queue to avoid polling and locking, which we feel would reduce load on the DB and simplify the implementation of the consumer code.
I对ActiveMQ，RabbitMQ和HornetQ做了一些研究，但我没有看到一个明显的方法来实现这一点。
I've done some research into ActiveMQ, RabbitMQ, and HornetQ but I don't see an obvious way to implement this.
ActiveMQ支持消费者订阅通配符，如果我可以这样做，那么解决方案就很简单：
ActiveMQ supports consumer subscription wildcards, but I don't see a way to limit the concurrency on each queue to 1. If I could do that, then the solution would be straightforward:
- 不知怎的告诉broker以允许所有队列开始并发为1：/ queue / person。
- 发布者使用队列名称中的Person ID将事件写入队列。例如：/queue/person.20
- 消费者使用通配符订阅队列：/ queue / person。>
- 每个消费者都会收到消息不同的人排队。如果所有人队列都在使用中，则一些消费者可以闲置，这是确定的
- 在处理消息之后，消费者发送ACK，告诉代理它完成消息，
- Somehow tell broker to allow a concurrency of 1 for all queues starting with: /queue/person.
- Publisher writes event to queue using Person ID in the queue name. e.g.: /queue/person.20
- Consumers subscribe to the queue using wildcards: /queue/person.>
- Each consumer would receive messages for different person queues. If all person queues were in use, some consumers may sit idle, which is ok
- After processing a message, the consumer sends an ACK, which tells the broker it's done with the message, and allows another message for that Person queue to be sent to another consumer (possibly the same one)
ActiveMQ已关闭：您可以执行通配符订阅并启用独占消费者，但是该组合导致单个消费者接收发送到所有匹配队列的所有消息，从而将所有人员的并发性降低到1。我觉得我缺少一些显而易见的东西。
ActiveMQ came close: You can do wildcard subscriptions and enable "exclusive consumer", but that combination results in a single consumer receiving all messages sent to all matching queues, reducing your concurrency to 1 across all Persons. I feel like I'm missing something obvious.
问题：
- 有什么方法来实现上述方法与任何主要的消息队列实现？我们相当开放的选择。唯一的要求是它在Linux上运行。
- 有没有不同的方法来解决我不考虑的一般问题？
谢谢！
推荐答案
看起来像JMSXGroupID是我正在寻找的。从ActiveMQ文档：
It looks like JMSXGroupID is what I'm looking for. From the ActiveMQ docs:
http：/ /activemq.apache.org/message-groups.html
他们的股票价格使用案例正是我所处理的。我唯一关心的是，如果单个消费者死亡会发生什么。希望代理将检测到并选择另一个消费者与该组ID相关联。
Their example use case with stock prices is exactly what I'm after. My only concern is what happens if the single consumer dies. Hopefully the broker will detect that and pick another consumer to associate with that group id.
这篇关于如何在保证每个实体的FIFO的同时处理消息？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
