Hadoop 与 SAS 的连接 [英] Hadoop connectivity with SAS

问题描述

我想使用 SAS/ACESS 9.3M2 接口将 sas 与我的 Hive 连接起来.我的问题是，sas 是否将 hive 立方体导入 sas 环境并在那里查询?或者，为了报告的目的，它再次命中 hive，因此它运行 MR，这将我的报告性能降低到 2-4 秒以上.

I want to use SAS/ACESS 9.3M2 Interface for connecting sas with my Hive. My question is, whether sas imports hive cubes into sas environment and queries there? or, It again hits hive for the purpose of reporting so that it runs MR which degrades my reporting performance to more than 2-4 secs.

如果将 hive 表导入其环境，与普通 sql 多维数据集相比，其性能如何?

If it imports hive tables to its environment what would be its performance when compared to normal sql cubes?

我对 sas 完全陌生，我希望我的报告在 2-4 秒内生成，其中我的聚合数据位于 Hive 表中，然后我在此之上创建了多维数据集维度.

I am totally new to sas i want my reports generated with in 2-4 secs where my aggregated data is in Hive tables and then I have created cube dimensions over that.

谢谢...

推荐答案

SAS/ACCESS 服务于:- 为您提供从/向数据源读取数据和写入数据的能力，处理数据类型转换- 提供有关数据存储的元数据(表、字段、数据类型的列表)- 提供一种(也部分)将(隐式传递)SAS代码转换为数据源特定代码(通常是SQL变体等)的方法- 为您提供一种编写数据源特定代码并将其从 SAS 发送到数据源中执行的方法

What SAS/ACCESS serves for is to: - provide you with ability to read data and write from/to a datasource, take care of data type conversions - provides metadata about a datastore (list of tables, fields, datatypes) - provide a mean to (also partially) translate (implicit pass-through) SAS code to datasource specific code (usually SQL variant etc) - provide a mean for you to write a datasource specific code and sent it from SAS for execution in datasource

我对 Hadoop 完全陌生 :-) 所以我只是猜测 SAS/访问 Hadoop(通过 LIBNAME 语句)从 Hadoop 读取关系数据，文档提到了 JDBC，所以我猜这是用于数据访问.我怀疑 SAS/Access 是否能够从 Hadoop 查询多维数据集(这是你的问题吗?- 我在上面创建了多维数据集维度" - 在 Hadoop 中是什么意思?).

I'm totally new to Hadoop :-) so I'll just guess that SAS/Access to Hadoop (via LIBNAME statement) reads relational data from Hadoop, the documentation mentions JDBC, so I guess that's used for data access. I'd doubt SAS/Access is able to query the cubes from Hadoop (is that your question? - "I have created cube dimensions over that" - meaning in Hadoop?).

通常 SAS/Access 会尽量减少来自数据源的数据传输，并尝试将处理推送到数据源.

Generally SAS/Access tries to minimize data transfers from datasources and tries to push the processing to the datasource.

来自 http:///blog.cloudera.com/blog/2013/05/how-the-sas-and-cloudera-platforms-work-together:

SAS/访问 Hadoop

SAS/ACCESS to Hadoop

SAS/ACCESS 提供了在 SAS 中访问存储在 Hadoop 中的数据集的能力.使用 SAS/访问 Hadoop:

SAS/ACCESS provides the ability to access data sets stored in Hadoop in SAS natively. With SAS/Access to Hadoop:

LIBNAME statements can be used to make Hive tables look like SAS data sets on top of which SAS Procedures and SAS DATA steps can interact.
PROC SQL commands provide the ability to execute direct Hive SQL commands on Hadoop.
PROC HADOOP provides the ability to directly submit MapReduce, Apache Pig, and HDFS commands from the SAS execution environment to your CDH cluster.

SAS/ACCESS 接口从 SAS 9.3M2 版本开始提供，并支持 CDH 3U2 以及 CDH 4.01 及更高版本.

The SAS/ACCESS interface is available from the SAS 9.3M2 release and supports CDH 3U2 as well as CDH 4.01 and higher.

PROC HADOOP 也可能有帮助http://support.sas.com/documentation/cdl/en/proc/65145/HTML/default/viewer.htm#p1esotuxnkbuepn1w443ueufw8in.htm

Also might be helpful PROC HADOOP at http://support.sas.com/documentation/cdl/en/proc/65145/HTML/default/viewer.htm#p1esotuxnkbuepn1w443ueufw8in.htm

这篇关于Hadoop 与 SAS 的连接的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！