pyspark是否支持spark-streaming-kafka-0-10 lib? [英] does pyspark support spark-streaming-kafka-0-10 lib?

问题描述

我的kafka集群版本为0.10.0.0，我想使用pyspark流读取kafka数据.但在Spark Streaming + Kafka集成指南中， http://spark.apache.org/docs/latest/streaming-kafka-0-10-integration.html 没有python代码示例. pyspark可以使用spark-streaming-kafka-0-10集成kafka吗?

my kafka cluster version is 0.10.0.0, and i want to use pyspark stream to read kafka data. but in Spark Streaming + Kafka Integration Guide, http://spark.apache.org/docs/latest/streaming-kafka-0-10-integration.html there is no python code example. so can pyspark use spark-streaming-kafka-0-10 to integrate kafka?

在此先感谢您的帮助！

推荐答案

当您的经纪人为0.10或更高版本时，可以使用spark-streaming-kafka-0-8. spark-streaming-kafka-0-8支持较新的代理版本，而streaming-kafka-0-10不支持较旧的代理版本.截至目前，streaming-kafka-0-10仍处于试验阶段，尚无Python支持.

You can use spark-streaming-kafka-0-8 when your brokers are 0.10 and later. spark-streaming-kafka-0-8 supports newer brokers versions while streaming-kafka-0-10 does not support older broker versions. streaming-kafka-0-10 as of now is still experimental and has no Python support.

这篇关于pyspark是否支持spark-streaming-kafka-0-10 lib?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！