跨微服务的数据一致性 [英] Data Consistency Across Microservices

问题描述

虽然每个微服务通常都会拥有自己的数据-但某些实体在多个服务之间必须保持一致.

While each microservice generally will have its own data - certain entities are required to be consistent across multiple services.

对于诸如微服务架构之类的高度分布式环境中的数据一致性要求，设计有哪些选择?当然，我不希望使用共享数据库体系结构，在该体系结构中，单个数据库管理所有服务的状态.这违反了隔离和无共享原则.

For such data consistency requirement in a highly distributed landscape such as microservices architecture, what are the choices for design? Of course, I do not want shared database architecture, where a single DB manages the state across all the services. That violates isolation and shared-nothing principles.

我确实知道，当实体被创建，更新或删除时，微服务可以发布事件.所有对此事件感兴趣的其他微服务都可以相应地更新其各自数据库中的链接实体.

I do understand that, a microservice can publish an event when an entity is created, updated or deleted. All other microservices which are interested in this event can accordingly update the linked entities in their respective databases.

这是可行的，但是会导致跨服务进行大量仔细而协调的编程工作.

This is workable, however it leads to a lot of careful and coordinated programming effort across the services.

Akka或任何其他框架可以解决此用例吗?怎么样?

Can Akka or any other framework solve this use case? How?

为了清楚起见，添加了下图.
基本上，我正在尝试了解是否存在可用的框架来解决此数据一致性问题.

Adding the below diagram for clarity.
Basically, I am trying to understand, if there are available frameworks today that can solve this data consistency problem.

对于队列，我可以使用任何AMQP软件，例如RabbitMQ或Qpid等. 对于数据一致性框架，我不确定目前Akka或任何其他软件是否可以提供帮助.还是这种情况如此罕见，而这种反模式却根本不需要任何框架?

For the queue I can use any AMQP software such as RabbitMQ or Qpid etc. For the data consistency framework, I am not sure if presently Akka or any other software can help. Or is this scenario so uncommon, and such an anti-pattern that no framework should be ever needed?

推荐答案

Microservices体系结构样式试图允许组织的小型团队拥有独立于开发和运行时的服务.请参阅此阅读.最困难的部分是以一种有用的方式定义服务边界.当您发现拆分应用程序的方式导致需求频繁影响多个服务时，这些需求将告诉您重新考虑服务边界.当您强烈需要在服务之间共享实体时，也是如此.

The Microservices architectural style tries to allow organizations to have small teams own services independent in development and at runtime. See this read. And the hardest part is to define the service boundaries in a useful way. When you discover that the way you split up your application results in requirements impacting multiple services frequently that would tell you to rethink the service boundaries. The same is true for when you feel a strong need to share entities between the services.

因此，一般建议是非常努力地避免这种情况.但是，在某些情况下您可能无法避免这种情况.由于好的架构通常是要进行正确的权衡，因此这里有一些想法.

So the general advice would be to try very hard to avoid such scenarios. However there may be cases where you cannot avoid this. Since a good architecture is often about making the right trade-offs, here some ideas.

1. 考虑使用服务接口(API)而不是直接DB依赖关系来表达依赖关系.这样一来，每个服务团队都可以根据需要更改其内部数据架构，并且仅在涉及依赖项时才担心接口设计.这很有用，因为添加新的API和缓慢地淘汰旧的API较容易，而不是更改数据库设计以及所有相关的微服务(可能同时).换句话说，只要仍然支持旧的API，您仍然能够独立部署新的Microservice版本.这是Amazon CTO推荐的方法，该公司率先开发了许多微服务方法.以下是推荐阅读的 2006年与他的访谈.

每当您真的无法避免使用相同的数据库并且以多个团队/服务需要相同实体的方式划分服务边界时，就会在微服务团队和负责团队之间引入两个依赖关系对于数据方案:a)数据格式，b)实际数据.这不是不可能解决的，而只是在组织中有一些开销.而且，如果引入过多的此类依赖关系，您的组织可能会瘫痪并减慢其发展速度.

Whenever you really really cannot avoid using the same DBs and you are splitting your service boundaries in a way that multiple teams/services require the same entities, you introduce two dependencies between the Microservice team and the team that is responsible for the data scheme: a) Data Format, b) Actual Data. This is not impossible to solve, but only with some overhead in organization. And if you introduce too many of such dependencies your organization will likely be crippled and slowed down in development.

a)对数据方案的依赖性.无需更改微服务，就无法修改实体数据格式.为了消除这种耦合，您将必须严格地 对实体数据方案进行版本控制，并且在数据库中支持微服务当前正在使用的所有数据版本.这将允许微服务团队自行决定何时更新其服务以支持新版本的数据方案.这并非在所有用例中都是可行的，但可以在许多用例中使用.

a) Dependency on the data scheme. The entities data format cannot be modified without requiring changes in the Microservices. To decouple this you will have to version the entities data scheme strictly and in the database support all versions of the data that the Microservices are currently using. This would allow the Microservices teams to decide for themselves when to update their service to support the new version of the data scheme. This is not feasible with all use cases, but it works with many.

b)依赖于实际收集的数据.可以使用已收集且具有微服务已知版本的数据，但是当您有一些服务在生成微服务时会出现此问题.数据的较新版本以及其他服务都取决于它-但尚未升级为能够读取最新版本.这个问题很难解决，并且在许多情况下表明您没有正确选择服务边界.通常，您别无选择，只能在升级数据库中的数据的同时推出所有依赖于数据的服务.一种更古怪的方法是同时写入不同版本的数据(通常在数据不可更改的情况下有效).

b) Dependency on the actual collected data. The data that has been collected and is of a known version for a Microservice is OK to use, but the issue occurs when you have some services producing a newer version of the data and another service depends on it - But was not yet upgraded to being able to read the latest version. This problem is hard to solve and in many cases suggests you did not chose the service boundaries correctly. Typically you have no choice but to roll out all services that depend on the data at the same time as upgrading the data in the database. A more wacky approach is to write different versions of the data concurrently (which works mostly when the data is not mutable).

在某些其他情况下，要解决a)和b)的问题，可以通过hidden data duplication和eventual consistency减少依赖关系.意味着每个服务都存储自己的数据版本，并且仅在该服务的需求发生变化时才对其进行修改.这些服务可以通过侦听公共数据流来实现.在这种情况下，您将使用基于事件的体系结构，其中定义了一组公共事件，这些事件可以由来自不同服务的侦听器排队和使用，这些服务将处理事件并从事件中存储与事件相关的任何数据(可能会造成数据重复).现在，其他一些事件可能表明必须更新内部存储的数据，并且每个服务都有责任使用自己的数据副本进行更新.维持这种公共事件队列的技术是 Kafka .

To solve both a) and b) in some other cases the dependency can be reduced by hidden data duplication and eventual consistency. Meaning each service stores its own version of the data and only modifies it whenever the requirements for that service change. The services can do so by listening to a public data flow. In such scenarios you would be using an event based architecture where you define a set of public events that can be queued up and consumed by listeners from the different services that will process the event and store whatever data out of it that is relevant for it (potentially creating data duplication). Now some other events may indicate that internally stored data has to be updated and it is each services responsibility to do so with its own copy of the data. A technology to maintain such a public event queue is Kafka.

这篇关于跨微服务的数据一致性的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！