微服务数据复制模式 [英] Microservice data replication patterns

问题描述

在微服务架构中，我们通常有两种方式来通信2个微服务.假设服务A需要从服务B获取信息.第一个选项是远程调用，通常是通过HTTPS同步的，因此服务A查询由服务B托管的API.

In a microservice architecture, we usually have two ways to communicate 2 microservices. Let’s say service A needs to get information from service B. The first option is an remote call, usually synchronous over HTTPS, so service A query an API hosted by service B.

第二种选择是采用事件驱动的体系结构，其中服务B的状态可以由服务A以异步方式发布和使用.使用此模型，服务A可以使用服务B的事件信息更新其自己的数据库，并且所有查询都在该数据库中本地进行.从开发到运营，这种方法都具有更好地分离微服务的优势.但是它具有与数据复制相关的一些缺点.

The second option is adopting an event-driven architecture, where the state of service B can be published and consumed by service A in asynchronous way. Using this model, the service A can update its own database with the information from the service B’s events and all queries are made locally in this database. This approach has the advantage of a better decoupling of microservices, since development until operations. But it comes with some disadvantages related to data replication.

第一个是磁盘空间的高消耗，因为相同的数据可以驻留在需要它的微服务的数据库中.但是第二个是最糟糕的:如果服务B无法按需要快速处理其订阅，或者服务B在服务B创建的同时不能用于服务A，则数据可能会过时.最终实现模型的一致性.

The first one is the high consumption of disk space, since the same data can reside in the databases of the microservices that need it. But the second one is worst in my opinion: data can become stale if service B can’t process its subscription as fast as needed, or it can’t be available for service A at the same time it’s created at service B, given the eventually consistency of the model.

假设我们将Kafka用作活动中心，其主题配置为保留7天的数据.服务A在服务B发布状态时保持同步.两周后，将部署一个新的服务C，并且需要使用服务B拥有的所有信息来丰富其数据库.由于最古老的事件已经过去，因此我们只能从Kafka主题中获取部分信息.我的问题是我们可以用来实现微服务数据库丰富化的模式(除了要求服务B将其所有当前状态重新发布到事件中心之外).

Let’s say we’re using Kafka as a event hub, and its topics are configured to use 7 days of data retention. Service A is kept in sync as service B publishes it state. After two weeks, a new service C is deployed and its database needs to be enriched with all information that service B holds. We can only get partial information from Kafka topics since the oldest events are gone. My question here is what are the patterns we can use to achieve this microservice’s database enrichment (besides asking service B to republish all its current state to the event hub).

推荐答案

有2个选项:

1. 您可以为单个主题启用Kafka的日志压缩.这样将保留给定密钥的最新值，丢弃旧的更新.在给定的保留期限内，这可以节省空间并且比普通模式保存更多的数据

1. You can enable log compaction for Kafka for an individual topic. That will keep the most recent value for a given key discarding old updates. This saves space and also holds more data than the normal mode for a given retention period

假设您每天都对服务B DB进行备份，那么在引入新服务C时，您需要首先从最新的B备份中创建C的初始状态，然后重播Kafka主题来自特定偏移ID的事件，该偏移ID代表备份后的数据.

Assuming you take a backup of service B DB on a daily basis, on introduction of a new service C, you need to first create the initial state of C from the latest backup of B and then replay the Kafka topic events from the particular offset id that represents the data after the backup.

这篇关于微服务数据复制模式的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！