Spark 2.0.0 Twitter流驱动程序不再可用 [英] Spark 2.0.0 twitter streaming driver is no longer available

问题描述

在从spark 1.6.2迁移到spark 2.0.0的过程中，出现了org.apache.spark.streaming.twitter软件包已被删除，twitter流以及依赖项不再可用的情况.

During migration from spark 1.6.2 to spark 2.0.0 appeared that package org.apache.spark.streaming.twitter has been removed and twitter streaming is no longer available as well as dependency

<dependency>
  <groupId>org.apache.spark</groupId>
  <artifactId>spark-streaming-twitter_2.11</artifactId>
  <version>2.0.0</version>
</dependency>

有人可以建议如何以新的方式处理Twitter流吗?

Can anyone suggest how to procced twitter stream in new spark?

推荐答案

Twitter(和其他一些)驱动程序支持已在Spark 2.0中删除.

Twitter (and some other) driver support has been removed in Spark 2.0.

您可以在发布的删除部分中看到它.注释:

移除

Spark 2.0中已删除以下功能:

• 较少使用的流媒体连接器，包括Twitter，Akka， MQTT，ZeroMQ

Removals

The following features have been removed in Spark 2.0:

• Less frequently used streaming connectors, including Twitter, Akka, MQTT, ZeroMQ

已将它们作为单独的软件包提取到 Bahir项目下. Twitter扩展 streaming-twitter 可以通过以下方式找到:

They have been extracted as a separate package under the Bahir Project. The twitter extension, streaming-twitter, can be found via:

sbt:

libraryDependencies += "org.apache.bahir" %% "spark-streaming-twitter" % "2.0.0"

行家:

<dependency>
  <groupId>org.apache.bahir</groupId>
  <artifactId>spark-streaming-twitter_2.11</artifactId>
  <version>2.0.0-preview</version>
</dependency>

邮件列表

对于Spark 2.0.1，请使用:

For Spark 2.0.1, use:

libraryDependencies += "org.apache.bahir" %% "spark-streaming-twitter" % "2.0.1"

这篇关于Spark 2.0.0 Twitter流驱动程序不再可用的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！