数据非规范化如何与微服务模式一起工作? [英] How does data denormalization work with the Microservice Pattern?

问题描述

我刚刚阅读了微服务和PaaS体系结构上的一篇文章.在那篇文章中，作者大约说了三分之一，(在像疯了一样疯狂地归一化下):

重构数据库架构，并对所有内容进行规范化，以允许数据的完全分离和分区.也就是说，请勿使用为多个微服务提供服务的基础表.不应共享跨越多个微服务的基础表，也不应共享数据.相反，如果多个服务需要访问相同的数据，则应通过服务API(例如已发布的REST或消息服务接口)进行共享.

尽管这种声音在理论上很棒，但在实践中却有一些严重的障碍需要克服.其中最大的问题是，通常数据库是紧密耦合的，并且每个表与至少一个其他表具有 some 外键关系.因此，不可能将数据库划分为受 n 个微服务控制的 n 个子数据库.

所以我问:给一个完全由相关表组成的数据库，如何将其规范化为较小的片段(表组)，以便可以由单独的微服务控制这些片段?

例如，给定以下(相当小但示例)的数据库:

[users] table
=============
user_id
user_first_name
user_last_name
user_email

[products] table
================
product_id
product_name
product_description
product_unit_price

[orders] table
==============
order_id
order_datetime
user_id

[products_x_orders] table (for line items in the order)
=======================================================
products_x_orders_id
product_id
order_id
quantity_ordered

不要花太多时间来批判我的设计，我可以即时进行.对我而言，关键是将这个数据库分为3个微服务是合乎逻辑的:

1. UserService-用于在系统中添加用户；最终应该管理[users]表；和
2. ProductService-用于在系统中添加产品；最终应该管理[products]表；和
3. OrderService-用于在系统中添加订单；最终应该管理[orders]和[products_x_orders]表

但是，所有这些表彼此之间都具有外键关系.如果我们对它们进行非规范化并将其视为整体，它们将失去所有语义含义:

[users] table
=============
user_id
user_first_name
user_last_name
user_email

[products] table
================
product_id
product_name
product_description
product_unit_price

[orders] table
==============
order_id
order_datetime

[products_x_orders] table (for line items in the order)
=======================================================
products_x_orders_id
quantity_ordered

现在无法知道谁订购了什么，订购多少，订购了什么时间.

那么，本文是典型的学术性论文，还是这种非规范化方法在现实世界中具有实用性?如果是，那么它看起来像什么(在答案中使用我的示例的加分点)?

解决方案

这是主观的，但以下解决方案对我，我的团队和我们的数据库团队有效.

• 在应用程序层，微服务被分解为语义功能.
  • 例如Contact服务可能会CRUD联系人(有关联系人的元数据:姓名，电话号码，联系人信息等)
  • 例如User服务可能会欺骗具有登录凭据，授权角色等的用户.
  • 例如Payment服务可能会CRUD付款，并与Stripe等第三方PCI兼容服务一起在后台运行.
• 在数据库层，可以对表进行组织，但是开发人员/数据库/开发人员希望对表进行组织

问题与级联和服务边界有关:付款可能需要用户知道谁在付款.不必像这样对服务建模:

interface PaymentService {
    PaymentInfo makePayment(User user, Payment payment);
}

建模如下:

interface PaymentService {
    PaymentInfo makePayment(Long userId, Payment payment);
}

通过这种方式，仅属于其他微服务的实体在特定服务中通过ID(而不是对象引用)被引用.这允许DB表到处都具有外键，但是在应用程序层，外部"实体(即，生活在其他服务中的实体)可通过ID获得.这样可以防止级联对象脱离控制，并明确划分服务边界.

它确实引起的问题是它需要更多的网络呼叫.例如，如果我给每个Payment实体一个User引用，我可以通过一次调用就获得特定付款的用户:

User user = paymentService.getUserForPayment(payment);

但是按照我在这里的建议，您需要打两次电话:

Long userId = paymentService.getPayment(payment).getUserId();
User user = userService.getUserById(userId);

这可能会破坏交易.但是，如果您很聪明，可以实现缓存，并实现精心设计的微服务，这些微服务可以在每次调用50-100毫秒内做出响应，那么毫无疑问，这些额外的网络调用可以精心制作，以使 not 不会引起延迟.应用程序.

I just read an article on Microservices and PaaS Architecture. In that article, about a third of the way down, the author states (under Denormalize like Crazy):

Refactor database schemas, and de-normalize everything, to allow complete separation and partitioning of data. That is, do not use underlying tables that serve multiple microservices. There should be no sharing of underlying tables that span multiple microservices, and no sharing of data. Instead, if several services need access to the same data, it should be shared via a service API (such as a published REST or a message service interface).

While this sounds great in theory, in practicality it has some serious hurdles to overcome. The biggest of which is that, often, databases are tightly coupled and every table has some foreign key relationship with at least one other table. Because of this it could be impossible to partition a database into n sub-databases controlled by n microservices.

So I ask: Given a database that consists entirely of related tables, how does one denormalize this into smaller fragments (groups of tables) so that the fragments can be controlled by separate microservices?

For instance, given the following (rather small, but exemplar) database:

[users] table
=============
user_id
user_first_name
user_last_name
user_email

[products] table
================
product_id
product_name
product_description
product_unit_price

[orders] table
==============
order_id
order_datetime
user_id

[products_x_orders] table (for line items in the order)
=======================================================
products_x_orders_id
product_id
order_id
quantity_ordered

Don't spend too much time critiquing my design, I did this on the fly. The point is that, to me, it makes logical sense to split this database into 3 microservices:

1. UserService - for CRUDding users in the system; should ultimately manage the [users] table; and
2. ProductService - for CRUDding products in the system; should ultimately manage the [products] table; and
3. OrderService - for CRUDding orders in the system; should ultimately manage the [orders] and [products_x_orders] tables

However all of these tables have foreign key relationships with each other. If we denormalize them and treat them as monoliths, they lose all their semantic meaning:

[users] table
=============
user_id
user_first_name
user_last_name
user_email

[products] table
================
product_id
product_name
product_description
product_unit_price

[orders] table
==============
order_id
order_datetime

[products_x_orders] table (for line items in the order)
=======================================================
products_x_orders_id
quantity_ordered

Now there's no way to know who ordered what, in which quantity, or when.

So is this article typical academic hullabaloo, or is there a real world practicality to this denormalization approach, and if so, what does it look like (bonus points for using my example in the answer)?

解决方案

This is subjective but the following solution worked for me, my team, and our DB team.

• At the application layer, Microservices are decomposed to semantic function.
  • e.g. a Contact service might CRUD contacts (metadata about contacts: names, phone numbers, contact info, etc.)
  • e.g. a User service might CRUD users with login credentials, authorization roles, etc.
  • e.g. a Payment service might CRUD payments and work under the hood with a 3rd party PCI compliant service like Stripe, etc.
• At the DB layer, the tables can be organized however the devs/DBs/devops people want the tables organized

The problem is with cascading and service boundaries: Payments might need a User to know who is making a payment. Instead of modeling your services like this:

interface PaymentService {
    PaymentInfo makePayment(User user, Payment payment);
}

Model it like so:

interface PaymentService {
    PaymentInfo makePayment(Long userId, Payment payment);
}

This way, entities that belong to other microservices only are referenced inside a particular service by ID, not by object reference. This allows DB tables to have foreign keys all over the place, but at the app layer "foreign" entities (that is, entities living in other services) are available via ID. This stops object cascading from growing out of control and cleanly delineates service boundaries.

The problem it does incur is that it requires more network calls. For instance, if I gave each Payment entity a User reference, I could get the user for a particular payment with a single call:

User user = paymentService.getUserForPayment(payment);

But using what I'm suggesting here, you'll need two calls:

Long userId = paymentService.getPayment(payment).getUserId();
User user = userService.getUserById(userId);

This may be a deal breaker. But if you're smart and implement caching, and implement well engineered microservices that respond in 50 - 100 ms each call, I have no doubt that these extra network calls can be crafted to not incur latency to the application.

这篇关于数据非规范化如何与微服务模式一起工作?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！