火花查询执行时间 [英] spark query execution time
问题描述
我安装了本地hadoop单节点和配置单元，并且在hdfs中存储了一些配置单元表.然后，我用MySQL Metastore配置Hive.现在我安装了spark和Im在像这样的配置单元表上做一些查询(在scala中):
I have a local hadoop single node and hive installed and I have some hive tables stored in hdfs. Then I configure Hive with MySQL Metastore. And now I installed spark and Im doing some queries over hive tables like this (in scala):
var hiveContext = new org.apache.spark.sql.hive.HiveContext(sc)
result = hiveContext.sql("SELECT * FROM USERS");
result.show
您知道如何配置spark以显示查询的执行时间吗?因为默认情况下它没有显示.
Do you know how to configure spark to show to the execution time of the query? Because for default it is not showing..
推荐答案
使用spark.time().
var hiveContext = new org.apache.spark.sql.hive.HiveContext(sc)
result = hiveContext.sql("SELECT * FROM USERS");
spark.time(result.show)
var hiveContext = new org.apache.spark.sql.hive.HiveContext(sc)
result = hiveContext.sql("SELECT * FROM USERS");
spark.time(result.show)
