是否有用于事件驱动的 Kafka 消费者的 Python API? [英] Is there a Python API for event-driven Kafka consumer?

问题描述

我一直在尝试构建一个以 Kafka 作为唯一界面的 Flask 应用程序.出于这个原因，我希望有一个 Kafka 消费者，当相关主题的流中有新消息时触发，并通过将消息推回 Kafka 流进行响应.

I have been trying to build a Flask app that has Kafka as the only interface. For this reason, I want have a Kafka consumer that is triggered when there is new message in the stream of the concerned topic and respond by pushing messages back to the Kafka stream.

我一直在寻找类似 Spring 实现的东西:

I have been looking for something like the Spring implementation:

@KafkaListener(topics = "mytopic", groupId = "mygroup")
public void listen(String message) {
    System.out.println("Received Messasge in group mygroup: " + message);
}

我看过:

1. kafka-python
2. pykafka
3. confluent-kafka

但我找不到任何与 Python 中事件驱动的实现方式相关的内容.

But I couldn't find anything related to event-driven style of implementation in Python.

推荐答案

Kafka Consumer 必须不断轮询以从 broker 检索数据.

Kafka Consumer have to continuously poll to retrieve data from brokers.

Spring 为您提供了这个花哨的 API，但实际上，它在循环中调用 poll，并且仅在检索到记录时调用您的方法.

Spring gives you this fancy API but under the covers, it calls poll in a loop and only invokes your method when records have been retrieved.

您可以使用您提到的任何 Python 客户端轻松构建类似的内容.就像在 Java 中一样，这不是(大多数)Kafka 客户端直接公开的 API，而是由顶层提供的东西.这是您需要构建的东西.

You can easily build something similar with any of the Python clients you've mentioned. Like in Java, this is not an API directly exposed by (most) Kafka clients but instead something provided by a layer on top. It's something you need to build.

这篇关于是否有用于事件驱动的 Kafka 消费者的 Python API?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！