pyspark 是否支持 spark-streaming-kafka-0-10 lib? [英] does pyspark support spark-streaming-kafka-0-10 lib?

问题描述

我的kafka集群版本是0.10.0.0，想用pyspark流读取kafka数据.但在 Spark Streaming + Kafka 集成指南中，http:///spark.apache.org/docs/latest/streaming-kafka-0-10-integration.html没有python代码示例.那么pyspark可以使用spark-streaming-kafka-0-10来集成kafka吗?

my kafka cluster version is 0.10.0.0, and i want to use pyspark stream to read kafka data. but in Spark Streaming + Kafka Integration Guide, http://spark.apache.org/docs/latest/streaming-kafka-0-10-integration.html there is no python code example. so can pyspark use spark-streaming-kafka-0-10 to integrate kafka?

预先感谢您的帮助！

推荐答案

当你的 broker 是 0.10 及更高版本时，你可以使用 spark-streaming-kafka-0-8.spark-streaming-kafka-0-8 支持较新的代理版本，而 streaming-kafka-0-10 不支持较旧的代理版本.到目前为止，streaming-kafka-0-10 仍处于实验阶段，不支持 Python.

You can use spark-streaming-kafka-0-8 when your brokers are 0.10 and later. spark-streaming-kafka-0-8 supports newer brokers versions while streaming-kafka-0-10 does not support older broker versions. streaming-kafka-0-10 as of now is still experimental and has no Python support.

这篇关于pyspark 是否支持 spark-streaming-kafka-0-10 lib?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！