Kafka python API 是否支持流处理? [英] Does Kafka python API support stream processing?

问题描述

我在 Java 中使用过 Kafka Streams.我在 python 中找不到类似的 API.Apache Kafka 是否支持 Python 中的流处理?

I have used Kafka Streams in Java. I could not find similar API in python. Do Apache Kafka support stream processing in python?

推荐答案

Kafka Streams 仅作为 JVM 库提供，但至少有两个 Python 实现它

Kafka Streams is only available as a JVM library, but there are at least two Python implementations of it

• robinhood/faust(截至 2020 年未维护，但 被分叉)
• wintincode/winton-kafka-streams(似乎不需要维护)

• robinhood/faust (Not maintained as of 2020, but was forked)
• wintincode/winton-kafka-streams (appears not to be maintained)

理论上，您可以尝试使用 Jython 或 Py4j 来处理 JVM 实现，但可能需要做更多不必要的工作.

In theory, you could try playing with Jython or Py4j to work with the JVM implementation, but probably would require more work than necessary.

除了这些选项之外，您还可以尝试 Apache Beam、Flink 或 Spark，但它们每个都需要一个外部集群调度程序来扩展(和还需要安装 Java).

Outside of those options, you can also try Apache Beam, Flink or Spark, but they each require an external cluster scheduler to scale out (and also require a Java installation).

如果您对 HTTP 方法没问题，那么运行 KSQLDB 实例(同样，该服务器需要 Java)并使用内置 SQL 函数从 Python 调用其 REST 接口可以工作.但是，在那里构建自己的函数需要编写 Java 代码，我上次检查过.

If you are okay with HTTP methods, then running a KSQLDB instance (again, requiring Java for that server) and invoking its REST interface from Python with the built-in SQL functions can work. However, building your own functions there will requiring writing Java code, last I checked.

如果这些选项都不适合，那么您只能使用基本的消费者/生产者方法.

If none of those options are suitable, then you're stuck with the basic consumer/producer methods.

这篇关于Kafka python API 是否支持流处理?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！