Kafka 消息 VS REST 调用 [英] Kafka Msg VS REST Calls

问题描述

如今在微服务世界中，我在我的工作场所看到很多使用 kafka 消息传递的设计，当您可以使用微服务之间的 rest api 调用实现类似的结果时.从技术上讲，您可以完全停止使用 rest api 调用，而是使用 kafka 消息传递.我真的很想知道最佳实践，优缺点，微服务之间何时使用 api 调用，何时使用 kafka 消息传递.

Nowadays in microservice world, i’m seeing alot of design in my workplace that uses kafka messaging when you can achieve similar results using rest api calls between microservices. Technically you can stop using rest api calls altogether and instead use kafka messaging. I really want to know the best practice, pros and cons, when to use api calls between microsevices, when to use kafka messaging.

让我们举一个真实的例子:

Lets put a real life example:

我有库存服务和供应商服务.日常供应商服务调用供应商 API 来获取新项目，这些需要转移到库存服务中.项目数最多可达 10,000 个对象.

I have an inventory service and a vendor service. Everyday vendor service calls the vendor API to get new items and these need to be moved into inventory service. The number of items can be up to 10,000 objects.

对于这个用例，是否更好:

For this use case, is it better to :

1. 从供应商 API 获取新数据后，调用库存服务的 REST API 来存储新项目.

1. After getting new data from vendor API, call REST API of inventory service to store the new items.

从供应商 API 获取新数据后，将它们作为消息发送到 kafka 主题，供库存服务使用

After getting new data from vendor API, send them as message to a kafka topic, to be consumed by inventory service

你会选择哪种方式以及考虑什么

Which way would you choose and what is the consideration

推荐答案

Gist(对于那些只想要 Gist 的人)

1. • Kafka - 发布 &订阅(只处理管道，工作完成后会通知)

1. • Kafka - Publish & Subscribe (just process the pipeline, will notify once the job is done)

REST - 请求 &等待响应(按需)

REST - Request & Await response (on-demand)

---

1. • Kafka - 发布一次 - 订阅 n 次(通过 n 个组件).

1. • Kafka - Publish once - Subscribe n times (by n components).

REST - 请求一次，得到一次响应.交易结束.

REST - Request once, get the response once. Deal over.

---

1. • Kafka - 数据存储在主题中.寻回&随时(偏移)直到主题被保留.

1. • Kafka - Data is stored in topic. Seek back & forth (offsets) whenever you want till the topic is retained.

REST - 一旦响应结束，它就结束了.手动使用数据库来存储处理后的数据.

REST - Once the response is over, it is over. Manually employ a database to store the processed data.

---

1. • Kafka - 拆分处理，将中间数据存储在中间主题中(为了速度和容错)

1. • Kafka - Split the processing, have intermediate data stored in intermediate topics (for speed and fault-tolerance)

REST - 获取数据，一次性处理所有数据，或者如果您希望将其分解，请不要忘记照顾好您自己的中间数据存储.

REST - Take the data, process it all at once OR if you wish to break it down, don't forget to take care of your OWN intermediate data stores.

---

1. • Kafka - 提出请求的人通常对响应不感兴趣(除了如果发送消息的响应)

1. • Kafka - The one who makes the request typically is not interested in a response (except the response that if the message is sent)

REST - 我提出请求意味着我通常期望得到回应(不仅仅是你收到的回应)请求，但对我有意义的东西，例如一些计算结果！)

REST - I am making the request means I typically expect a response (not just a response that you have received the request, but something that is meaningful to me, some computed result for example!)

问答风格

您的数据是否在流式传输?
如果数据不断涌入并且您有一个管道来执行，那么 Kafka 是最佳选择.

Q&A style

Is your data streaming?
If the data keeps on coming and you have a pipeline to execute, Kafka is best.

您需要请求-响应模型吗?
如果用户请求某事并等待响应，那么 REST 是最好的.

Do you need a request-response model?
If the user requests for something and they wait for a response, then REST is best.

Kafka(或任何其他流媒体平台)通常用于管道，即我们有前向流数据.

Kafka (or any other streaming platform) is typically used for pipelines i.e where we have forward flow of data.

数据进入 Kafka，然后通过 component1、 component2 等等，最后(通常)进入数据库.

Data comes to Kafka and from there it goes through component1, component2 and so on and finally (typically) lands in a database.

为了按需获取信息，我们需要一个数据存储(一个数据库)，我们可以在其中查询和获取信息.在这种情况下，我们提供了一个 REST 接口，用户可以调用它并获取他们想要的数据.

To get the information on-demand we need a data store (a database) where we can query and get it. In such a case we provide a REST interface which the user can invoke and get the data they want.

关于你的例子，

日常供应商服务调用供应商 API 以获取新商品和这些需要转移到库存服务中

Everyday vendor service calls the vendor API to get new items and these need to be moved into inventory service

问题&答案

您的供应商 API 是否使用 REST?

然后你需要拉数据并推送到Kafka.从那里您的库存服务(或之后的任何其他服务)将订阅该主题并执行其处理逻辑.

Then you need to pull the data and push to Kafka. From there your inventory service (or any other service thereafter) will subscribe to that topic and execute their processing logic.

这里的优点是您可以将需要供应商数据作为使用者的任何其他服务添加到供应商主题中.

此外，即使在您的库存服务处理之后，供应商数据也始终存在.

Moreover, the vendor data is always there for you even after your inventory service processed it.

如果为此使用 REST，则需要为每个需要供应商数据的组件调用供应商 API，这在与 Kafka 一起使用时变得微不足道

是否要查询库存?

通过 Kafka 处理后将其存储在数据库中，并在此基础上提供 REST.这是必需的，因为 Kafka 通常是一个日志，要使数据可查询，您需要一些数据库.

Store it in a database after processing through Kafka and provide a REST on top of this. This is needed because Kafka is typically a log, to make the data query-able you would need some database.

这篇关于Kafka 消息 VS REST 调用的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！