在 SparkSQL 中缓存表时出错 [英] Error in Caching a Table in SparkSQL
问题描述
我正在尝试缓存 Hive 中可用的表(使用 spark-shell
).下面给出的是我的代码
I am trying to cache a Table available in Hive(using spark-shell
). Given below is my code
scala> val hiveContext = new org.apache.spark.sql.hive.HiveContext(sc)
scala> hiveContext.cacheTable("sparkdb.firsttable")
我收到以下异常
org.apache.spark.sql.catalyst.analysis.NoSuchTableException
at org.apache.spark.sql.hive.client.ClientInterface$$anonfun$getTable$1.apply(ClientInterface.scala:112)
表 firsttable
在数据库 sparkdb
(在 Hive 中)中可用.看起来问题似乎在于提供数据库名称.我如何实现这一目标?
The table firsttable
is available in database sparkdb
(in Hive). Looks like the issue seems to be in providing database name. How do I achieve this?
PS:如下所示的 HiveQL 查询确实可以正常工作
PS : HiveQL query like the one shown below does work without any issues
scala>hiveContext.sql("select * from sparkdb.firsttable")
从其他几个方法调用中找出以下结果
Find below results from few other method calls
scala> hiveContext.tables("sparkdb")
res14: org.apache.spark.sql.DataFrame = [tableName: string, isTemporary: boolean]
scala> hiveContext.tables("sparkdb.firsttable")
res15: org.apache.spark.sql.DataFrame = [tableName: string, isTemporary: boolean]
推荐答案
啊哈!我是对的,这似乎是 SPARK-8105.因此,目前最好的办法是执行 select *
并缓存它.
Aha! I was right, this seems to be SPARK-8105. So, for now, your best bet is to do the select *
and cache that.
这篇关于在 SparkSQL 中缓存表时出错的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!