微服务架构-在订单无关紧要时通过服务传递消息 [英] Microservice architecture - carry message through services when order doesn't matter

问题描述

Tl; dr :我如何才能通过一堆异步，无序的微服务推送消息，并知道该消息何时通过它们中的每一个?"

我正在努力为特定的微服务体系结构找到正确的消息传递系统/协议.这不是一个最好的"问题，而是一个关于我对设计模式/协议的选择的问题.

• 我在开始队列中有一个消息.假设一个带有序列化JSON的RabbitMQ消息
• 我需要该消息才能通过任意数量的微服务
• 这些微服务中的每一个都是长期运行的，必须独立并且可以用多种语言实现
• 消息通过的服务顺序无关紧要.实际上，它不应该是同步的.
• 每个服务都可以将数据追加到原始消息中，但是其他服务将忽略该数据.应该没有合并错误(每个服务都写一个唯一的密钥).任何服务都不会更改或破坏数据.
• 一旦所有服务轮到他们了，则应将消息连同原始数据和新数据一起发布到第二个RabbitMQ队列.
• 微服务不会有其他副作用.如果所有这些都在一个整体应用程序中(并且使用相同的语言)，那么函数式编程将是完美的.

因此，问题是，通过各种服务来管理该消息的适当方法是什么?我不想一次只要做一次，订单并不重要.但是，如果是这样的话，系统如何知道何时所有服务都受到了重击，并且最终消息可以写到结束队列中(以使下一批服务得到使用).

我唯一能想到的半优雅的解决方案是

1. 要使第一个遇到消息的服务将该消息写入公共存储(例如mongodb)
2. 让每个服务执行其操作，标记已完成该消息，然后检查是否所有服务都轮到了
3. 如果是这样，则最后一个服务将发布该消息

但这仍然要求每个服务了解所有其他服务，并且 要求每个服务都留下自己的印记.这些都不是想要的.

我愿意接受某种牧羊人"服务.

我会很感激我错过的任何选择，并愿意承认它们可能是更好的基本设计.

谢谢.

解决方案

管理长期运行的流程(或涉及多个微服务的处理)有两种方法:编排和编排.有很多描述它们的文章.

长话短说:在业务流程中，您有一个微服务可以跟踪流程状态，而在编排中，所有微服务都知道在哪里接下来发送消息和/或完成该过程.

该文章解释了两种样式的优点和取舍.

编排

编排好处

• 提供了一种在进行同步处理时控制应用程序流的好方法.例如，如果服务A需要在调用服务B之前成功完成.

业务流程权衡

• 将服务耦合在一起，从而创建依赖关系.如果服务A关闭，则服务B和C将永远不会被调用.

• 如果所有请求的协调器都有一个中央共享实例，那么该协调器就是单点故障.如果出现故障，所有处理将停止.

• 利用阻止请求的同步处理.在此示例中，端到端的总处理时间是调用服务A +服务B +服务C所花费的时间的总和.

编舞

编舞的好处

• 由于可以并行/异步执行服务，因此可以实现更快的端到端处理.

• 更容易添加/更新服务，因为它们可以轻松插入事件流/从事件流中插入.

• 与敏捷交付模型非常吻合，因为团队可以专注于特定服务而不是整个应用程序.

• 控制是分布式的，因此不再有单个协调器成为故障的中心点.

• 几种模式可以与反应式体系结构一起使用，以提供其他好处.例如，事件来源是指事件流存储所有事件并启用事件重播的时间.这样，如果服务仍在生成事件时发生故障，则当服务重新联机时，它可以重播这些事件以进行恢复.此外，可以将命令查询责任隔离(CQRS)应用于读写活动.这使得它们中的每一个都可以独立缩放.如果您的应用程序是读取密集型的，并且写时比较轻便，反之亦然.

编排权衡

• 对于开发人员来说，异步编程通常是一个重要的转变.我倾向于认为它与递归类似，在递归中，您不能仅凭一眼就知道代码将如何执行，而必须考虑在特定时间点可能存在的所有可能性. p>

• 复杂性发生了变化.现在，不再将流程控制集中在协调器中，而是将流程控制分解并分布在各个服务中.每个服务都有其自己的流逻辑，并且该逻辑将根据事件流中的特定数据来确定何时以及如何响应.

Tl;dr: "How can I push a message through a bunch of asynchronous, unordered microservices and know when that message has made it through each of them?"

I'm struggling to find the right messaging system/protocol for a specific microservices architecture. This isn't a "which is best" question, but a question about what my options are for a design pattern/protocol.

• I have a message on the beginning queue. Let's say a RabbitMQ message with serialized JSON
• I need that message to go through an arbitrary number of microservices
• Each of those microservices are long running, must be independent, and may be implemented in a variety of languages
• The order of services the message goes through does not matter. In fact, it should not be synchronous.
• Each service can append data to the original message, but that data is ignored by the other services. There should be no merge conflicts (each service writes a unique key). No service will change or destroy data.
• Once all the services have had their turn, the message should be published to a second RabbitMQ queue with the original data and the new data.
• The microservices will have no other side-effects. If this were all in one monolithic application (and in the same language), functional programming would be perfect.

So, the question is, what is an appropriate way to manage that message through the various services? I don't want to have to do one at a time, and the order isn't important. But, if that's the case, how can the system know when all the services have had their whack and the final message can be written onto the ending queue (to have the next batch of services have their go).

The only, semi-elegant solution I could come up with was

1. to have the first service that encounters a message write that message to common storage (say mongodb)
2. Have each service do its thing, mark that it has completed for that message, and then check to see if all the services have had their turn
3. If so, that last service would publish the message

But that still requires each service to be aware of all the other services and requires each service to leave its mark. Neither of those is desired.

I am open to a "Shepherd" service of some kind.

I would appreciate any options that I have missed, and am willing to concede that their may be a better, fundamental design.

Thank you.

解决方案

There are two methods of managing a long running process (or a processing involving multiple microservices): Orchestration and choreography. There are a lot of articles describing them.

Long story short: In Orchestration you have a microservice that keeps track of the process status and in Choreography all the microservices know where to send next the message and/or when the process is done.

This article explains the benefits and tradeofs of the two styles.

Orchestration

Orchestration Benefits

• Provides a good way for controlling the flow of the application when there is synchronous processing. For example, if Service A needs to complete successfully before Service B is invoked.

Orchestration Tradeoffs

• Couples the services together creating dependencies. If service A is down, service B and C will never be called.

• If there is a central shared instance of the orchestrator for all requests, then the orchestrator is a single point of failure. If it goes down, all processing stops.

• Leverages synchronous processing that blocks requests. In this example, the total end-to-end processing time is the sum of time it takes for Service A + Service B + Service C to be called.

Choreography

Choreography Benefits

• Enables faster end-to-end processing as services can be executed in parallel/asynchronously.

• Easier to add/update services as they can be plugged in/out of the event stream easily.

• Aligns well with an agile delivery model as teams can focus on particular services instead of the entire application.

• Control is distributed, so there is no longer a single orchestrator serving as a central point of failure.

• Several patterns can be used with a reactive architecture to provide additional benefits. For example, Event Sourcing is when the Event Stream stores all of the events and enables event replay. This way, if a service went down while events were still being produced, when it came back online it could replay those events to catch back up. Also, Command Query Responsibility Segregation (CQRS) can be applied to separate out the read and write activities. This enables each of these to be scaled independently. This comes in handy if you have an application that is read-heavy and light on writes or vice versa.

Choreography Tradeoffs

• Async programming is often a significant mindshift for developers. I tend to think of it as similar to recursion, where you can’t figure out how code will execute by just looking at it, you have to think through all of the possibilities that could be true at a particular point in time.

• Complexity is shifted. Instead of having the flow control centralized in the orchestrator, the flow control is now broken up and distributed across the individual services. Each service would have its own flow logic, and this logic would identify when and how it should react based on specific data in the event stream.

这篇关于微服务架构-在订单无关紧要时通过服务传递消息的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！