如何使用星火与HiveContext [英] How to use Spark with HiveContext
问题描述
我是新来spark.I想使用HiveContext在sparkSQL访问窗口functions.But我给这个时候，
I am new to spark.I am trying to use HiveContext in sparkSQL to access window functions.But when I give this,
import org.apache.spark.sql.hive.HiveContext
我收到以下错误：
I get the following error:
错误：对象蜂巢不包org.apache.spark.sql成员
error: object hive is not a member of package org.apache.spark.sql
我发现我需要在build.sbt文件以下相关解决方案。
I found out the solution that I need to include the following dependencies in build.sbt file.
libraryDependencies += "org.apache.spark" %% "spark-hive" % "1.5.2
但我不知道，我应该在哪里找到这个火花文件build.sbt
我应该改变，
But I don't know,where should I find this build.sbt file in spark Should I change in,
/home/cloudera/spark-1.5.2/build/sbt
（或）
/home/cloudera/spark-1.5.2/sbt/sbt
（或）
/home/cloudera/spark-1.5.2/project/plugins.sbt
请帮我添加的依赖关系build.sbt文件。
Please help me to add the dependencies in build.sbt file.
推荐答案
要建立与蜂巢支持星火没有必要在构建定义的任何变化。只是确保您激活所需的型材
To build Spark with Hive support there is no need for any changes in build definitions. Just makes sure you activate required profiles
-
的Maven
Maven
mvn -Phive -Phive-thriftserver -DskipTests ... clean package
SBT
SBT
build/sbt -Phive -Phive-thriftserver ... assembly
添加火花蜂巢，因为只有当你建立自己的项目的依赖是必须的。
Adding spark-hive as a dependency is required only when you build your own projects.
另外，请不就是pre-建（火花[SPARK-版本] -bin-的Hadoop [Hadoop的版本] .tgz的文件）提供的二进制文件从星火下载页面已经建有蜂巢的支持。
Also, please not that pre-built (spark-[SPARK-VERSION]-bin-hadoop[HADOOP-VERSION].tgz files) binaries available from Spark downloads page are already built with Hive support.
这篇关于如何使用星火与HiveContext的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
