微服务架构 - 当订单无关紧要时通过服务传递消息 [英] Microservice architecture - carry message through services when order doesn't matter

问题描述

Tl;dr:我如何通过一堆异步、无序的微服务推送消息，并知道该消息何时通过每个微服务?"

我正在努力为特定的微服务架构寻找合适的消息传递系统/协议.这不是一个哪个最好"的问题，而是一个关于我对设计模式/协议的选择是什么的问题.

• 我在开始队列中有一条消息.假设一个带有序列化 JSON 的 RabbitMQ 消息
• 我需要该消息通过任意数量的微服务
• 这些微服务中的每一个都是长期运行的，必须是独立的，并且可以用多种语言实现
• 消息通过的服务顺序无关紧要.事实上，它不应该是同步的.
• 每个服务都可以将数据附加到原始消息中，但其他服务会忽略该数据.应该有没有合并冲突(每个服务写入一个唯一的密钥).任何服务都不会更改或破坏数据.
• 一旦轮到所有服务，消息应该与原始数据和新数据一起发布到第二个 RabbitMQ 队列.
• 微服务不会有其他副作用.如果这一切都在一个单一的应用程序中(并且使用相同的语言)，那么函数式编程将是完美的.

那么，问题是，通过各种服务管理该消息的适当方法是什么?我不想一次只做一个，顺序并不重要.但是，如果是这样的话，系统怎么知道所有的服务什么时候都完成了，最后的消息可以被写入结束队列(让下一批服务开始).

我能想到的唯一、半优雅的解决方案是

1. 让第一个遇到消息的服务将该消息写入公共存储(例如 mongodb)
2. 让每个服务都做自己的事情，标记它已针对该消息完成，然后检查是否所有服务都轮到他们了
3. 如果是这样，最后一个服务将发布消息

但这仍然要求每个服务都了解所有其他服务并且要求每个服务留下自己的标记.这些都不是我们想要的.

我愿意接受某种牧羊人"服务.

我会感谢我错过的任何选项，并愿意承认它们可能是更好的基本设计.

谢谢.

解决方案

管理长时间运行的进程(或涉及多个微服务的处理)有两种方法:编排和编排.有很多文章描述它们.

长话短说:在 Orchestration 中，您有一个跟踪流程状态的微服务，而在 Choreography 中，所有微服务都知道该去哪里发送下一条消息和/或在该过程完成时发送.

这篇

编排优势

• 提供了一种在同步处理时控制应用流程的好方法.例如，如果服务 A 需要在调用服务 B 之前成功完成.

协调权衡

• 将服务耦合在一起，创建依赖关系.如果服务 A 宕机，服务 B 和 C 将永远不会被调用.

• 如果所有请求都有一个协调器的中央共享实例，那么协调器就是单点故障.如果它出现故障，所有处理都会停止.

• 利用阻止请求的同步处理.在本例中，端到端的总处理时间是调用服务 A + 服务 B + 服务 C 所用时间的总和.

编舞

编排优势

• 支持更快的端到端处理，因为服务可以并行/异步执行.

• 更容易添加/更新服务，因为它们可以轻松插入/退出事件流.

• 非常适合敏捷交付模型，因为团队可以专注于特定服务而不是整个应用程序.

• 控制是分布式的，因此不再有单个协调器充当故障中心点.

• 多种模式可以与反应式架构一起使用，以提供额外的好处.例如，事件溯源是指事件流存储所有事件并启用事件重放.这样，如果服务在事件仍在产生时出现故障，当它重新上线时，它可以重播这些事件以赶上.此外，可以应用命令查询职责分离 (CQRS) 来分离读取和写入活动.这使得这些中的每一个都可以独立缩放.如果您的应用程序读取量大而写入量少，反之亦然，这会派上用场.

编排权衡

• 异步编程通常是开发人员的重要思维转变.我倾向于认为它类似于递归，您无法仅通过查看代码就知道代码将如何执行，您必须考虑在特定时间点可能存在的所有可能性.

• 复杂性被转移了.流程控制不再集中在编排器中，而是现在被分解并分布在各个服务中.每个服务都有自己的流逻辑，该逻辑将根据事件流中的特定数据确定何时以及如何做出反应.

Tl;dr: "How can I push a message through a bunch of asynchronous, unordered microservices and know when that message has made it through each of them?"

I'm struggling to find the right messaging system/protocol for a specific microservices architecture. This isn't a "which is best" question, but a question about what my options are for a design pattern/protocol.

• I have a message on the beginning queue. Let's say a RabbitMQ message with serialized JSON
• I need that message to go through an arbitrary number of microservices
• Each of those microservices are long running, must be independent, and may be implemented in a variety of languages
• The order of services the message goes through does not matter. In fact, it should not be synchronous.
• Each service can append data to the original message, but that data is ignored by the other services. There should be no merge conflicts (each service writes a unique key). No service will change or destroy data.
• Once all the services have had their turn, the message should be published to a second RabbitMQ queue with the original data and the new data.
• The microservices will have no other side-effects. If this were all in one monolithic application (and in the same language), functional programming would be perfect.

So, the question is, what is an appropriate way to manage that message through the various services? I don't want to have to do one at a time, and the order isn't important. But, if that's the case, how can the system know when all the services have had their whack and the final message can be written onto the ending queue (to have the next batch of services have their go).

The only, semi-elegant solution I could come up with was

1. to have the first service that encounters a message write that message to common storage (say mongodb)
2. Have each service do its thing, mark that it has completed for that message, and then check to see if all the services have had their turn
3. If so, that last service would publish the message

But that still requires each service to be aware of all the other services and requires each service to leave its mark. Neither of those is desired.

I am open to a "Shepherd" service of some kind.

I would appreciate any options that I have missed, and am willing to concede that their may be a better, fundamental design.

Thank you.

解决方案

There are two methods of managing a long running process (or a processing involving multiple microservices): Orchestration and choreography. There are a lot of articles describing them.

Long story short: In Orchestration you have a microservice that keeps track of the process status and in Choreography all the microservices know where to send next the message and/or when the process is done.

This article explains the benefits and tradeofs of the two styles.

Orchestration

Orchestration Benefits

• Provides a good way for controlling the flow of the application when there is synchronous processing. For example, if Service A needs to complete successfully before Service B is invoked.

Orchestration Tradeoffs

• Couples the services together creating dependencies. If service A is down, service B and C will never be called.

• If there is a central shared instance of the orchestrator for all requests, then the orchestrator is a single point of failure. If it goes down, all processing stops.

• Leverages synchronous processing that blocks requests. In this example, the total end-to-end processing time is the sum of time it takes for Service A + Service B + Service C to be called.

Choreography

Choreography Benefits

• Enables faster end-to-end processing as services can be executed in parallel/asynchronously.

• Easier to add/update services as they can be plugged in/out of the event stream easily.

• Aligns well with an agile delivery model as teams can focus on particular services instead of the entire application.

• Control is distributed, so there is no longer a single orchestrator serving as a central point of failure.

• Several patterns can be used with a reactive architecture to provide additional benefits. For example, Event Sourcing is when the Event Stream stores all of the events and enables event replay. This way, if a service went down while events were still being produced, when it came back online it could replay those events to catch back up. Also, Command Query Responsibility Segregation (CQRS) can be applied to separate out the read and write activities. This enables each of these to be scaled independently. This comes in handy if you have an application that is read-heavy and light on writes or vice versa.

Choreography Tradeoffs

• Async programming is often a significant mindshift for developers. I tend to think of it as similar to recursion, where you can’t figure out how code will execute by just looking at it, you have to think through all of the possibilities that could be true at a particular point in time.

• Complexity is shifted. Instead of having the flow control centralized in the orchestrator, the flow control is now broken up and distributed across the individual services. Each service would have its own flow logic, and this logic would identify when and how it should react based on specific data in the event stream.

这篇关于微服务架构 - 当订单无关紧要时通过服务传递消息的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！