微服务数据复制模式 [英] Microservice data replication patterns

问题描述

在微服务架构中，我们通常有两种微服务通信方式.假设服务 A 需要从服务 B 获取信息.第一个选项是远程调用，通常通过 HTTPS 同步，因此服务 A 查询服务 B 托管的 API.

In a microservice architecture, we usually have two ways for 2 microservices to communicate. Let’s say service A needs to get information from service B. The first option is a remote call, usually synchronous over HTTPS, so service A query an API hosted by service B.

第二种选择是采用事件驱动的架构，其中服务 B 的状态可以以异步方式由服务 A 发布和消费.使用此模型，服务 A 可以使用来自服务 B 的事件的信息更新自己的数据库，并且所有查询都在此数据库中本地进行.这种方法的优点是可以更好地分离微服务，从开发到运营.但它带来了一些与数据复制相关的缺点.

The second option is adopting an event-driven architecture, where the state of service B can be published and consumed by service A in an asynchronous way. Using this model, service A can update its own database with the information from the service B’s events and all queries are made locally in this database. This approach has the advantage of a better decoupling of microservices, from development until operations. But it comes with some disadvantages related to data replication.

第一个是磁盘空间的高消耗，因为相同的数据可以驻留在需要它的微服务的数据库中.但在我看来，第二个最糟糕:如果服务 B 不能按需要快速处理其订阅，或者在服务 B 上创建它的同时它不能用于服务 A，则数据可能会变得陈旧，因为模型的最终一致性.

The first one is the high consumption of disk space, since the same data can reside in the databases of the microservices that need it. But the second one is worst in my opinion: data can become stale if service B can’t process its subscription as fast as needed, or it can’t be available for service A at the same time it’s created at service B, given the eventual consistency of the model.

假设我们使用 Kafka 作为事件中心，其主题配置为使用 7 天的数据保留期.当服务 B 发布其状态时，服务 A 保持同步.两周后，新服务 C 部署完毕，其数据库需要使用服务 B 拥有的所有信息进行充实.由于最旧的事件已经消失，我们只能从 Kafka 主题中获取部分信息.我的问题是我们可以使用哪些模式来实现此微服务的数据库丰富(除了要求服务 B 将其所有当前状态重新发布到事件中心).

Let’s say we’re using Kafka as an event hub, and its topics are configured to use 7 days of data retention. Service A is kept in sync as service B publishes its state. After two weeks, a new service C is deployed and its database needs to be enriched with all information that service B holds. We can only get partial information from Kafka topics since the oldest events are gone. My question here is what are the patterns we can use to achieve this microservice’s database enrichment (besides asking service B to republish all its current state to the event hub).

推荐答案

有 2 个选项:

1. 您可以为单个主题启用 Kafka 的日志压缩.这将保留给定密钥的最新值，丢弃旧更新.在给定的保留期限内，这可以节省空间并比正常模式保存更多数据

1. You can enable log compaction for Kafka for an individual topic. That will keep the most recent value for a given key discarding old updates. This saves space and also holds more data than the normal mode for a given retention period

假设您每天备份服务B DB，在引入新服务C时，您需要首先从B的最新备份中创建C的初始状态，然后重放Kafka主题来自代表备份后数据的特定偏移 id 的事件.

Assuming you take a backup of service B DB on a daily basis, on introduction of a new service C, you need to first create the initial state of C from the latest backup of B and then replay the Kafka topic events from the particular offset id that represents the data after the backup.

这篇关于微服务数据复制模式的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！