Kafka 是否支持请求响应消息传递 [英] Does Kafka support request response messaging

问题描述

我正在研究 Kafka 9 作为一个业余项目，并完成了一些Hello World"类型的示例.

I am investigating Kafka 9 as a hobby project and completed a few "Hello World" type examples.

我必须考虑基于请求响应消息的真实世界 Kafka 应用程序，更具体地说，如何将 Kafka 请求消息链接到其响应消息.

I have got to thinking about Real World Kafka applications based on request response messaging in general and more specifically how to link a Kafka request message to its response message.

我正在考虑使用生成的 UUID 作为请求消息键，并使用此请求 UUID 作为关联的响应消息键.与 WebSphere MQ 具有消息关联 ID 的机制类型非常相似.

I was thinking along the lines of using a generated UUID as the request message key and employ this request UUID as the associated response message key. Much the same type of mechanism that WebSphere MQ has message correlation id.

我的 end 2 end 流程是.

My end 2 end process would be.

1).Kafka 客户端生成一个随机 UUID 并发送单个 Kafka 请求消息.2).服务器将使用该请求消息提取&存储请求 UUID 值3).使用消息有效负载完成业务流程.4).使用响应消息响应，该响应消息使用请求消息中存储的 UUID 值作为响应消息 Key.5).Kafka 客户端轮询响应主题，直到超时或检索到具有原始请求 UUID 值的消息.

1). Kafka client generates a random UUID and sends a single Kafka request message. 2). The server would consume this request message extract & store the request UUID value 3). complete a Business Process using the message payload. 4). Respond with a response message that employs the stored UUID value from the request message as response message Key. 5). the Kafka client polls the response topic until it either timeouts or retrieves a message with the original request UUID value.

我担心的是 Kafka Consumer 轮询会从响应主题中删除其他客户端消息并增加偏移量，从而导致其他客户端失败.

What I concerned about is that the Kafka Consumer polling will remove other clients messages from the response topic and increment the offsets making other clients fail.

我是否试图将 Kafka 应用到它从未设计过的用例中?

Am I trying to apply Kafka in a use case it was never designed for?

是否可以在 Kafka 中实现请求/响应消息传递?

Is it possible to implement request/response messaging in Kafka?

推荐答案

尽管 Kafka 提供了方便的方法来为给定的消费者组持久化已提交的偏移量，但您不需要使用该行为，如果您愿意，也可以编写自己的觉得需要.即便如此，按照您描述的方式使用 Kafka 对于用例来说还是有点尴尬，因为每个客户端都需要重复搜索主题以获取特定响应.这充其量是低效的.

Even though Kafka provides convenience methods to persist the committed offsets for a given consumer group, you're not required to use that behavior and can write your own if you feel the need. Even so, the use of Kafka the way you've described it is a bit awkward for the use case as each client needs to repeatedly search the topic for a specific response. That's inefficient at best.

您可以将问题分为两部分，继续使用 Kafka 向您的服务器发送请求和从您的服务器发送响应.您需要添加的唯一部分是您的客户端与之通信的某种 API 层，它封装了来自您的客户端的 Kafka 特定逻辑.这一层需要一个本地数据库(关系或 NoSQL)，它可以通过 uuid 存储响应，从而使 API 能够非常快速和轻松地回答响应是否可用于特定 uuid.

You could break the problem into two parts, continuing to use Kafka to deliver requests to and responses from your server. The only piece you'd need to add would be some sort of API layer that your clients talk to and which encapsulates the Kafka-specific logic from your clients. This layer would need a local DB (relational or NoSQL) that could store responses by uuid making it very fast and easy for the API to answer whether a response is available for a specific uuid.

这篇关于Kafka 是否支持请求响应消息传递的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！