“针对Apache Hadoop 2.7及更高版本预构建"的内容是什么?意思是? [英] What does "pre-built for Apache Hadoop 2.7 and later" mean?

问题描述

在Apache Spark的下载页面上，为Apache Hadoop 2.7和更高版本预构建"是什么意思?

What does "pre-built for Apache Hadoop 2.7 and later" mean on the download page of Apache Spark?

这是否意味着在火花中安装HDFS所需的库?如果是这样，那么其他存储系统(如Cassandra，s3，HBase，SQL数据库，NoSQL数据库)如何?我们是否需要下载任何库才能连接到其他存储系统.

Does it mean necessary libraries for HDFS in spark? If so, how about other storage systems like Cassandra, s3, HBase, SQL Databases, NoSQL databases? Do we need to download any libraries to connect to other storage systems.

推荐答案

这是否意味着在火花中安装HDFS所需的库

Does it mean necessary libraries for HDFS in spark

正确！Spark使用Hadoop FileSystem API来访问文件(在HDFS和S3以及其他受HDFS支持的文件系统上)，并且为Apache Hadoop 2.7和更高版本预构建"版本随附必需的库.

Correct! Spark uses Hadoop FileSystem API to access files (on HDFS and S3 and other HDFS-supported file systems) and "pre-built for Apache Hadoop 2.7 and later" version comes with the necessary libraries included.

主要用于Spark Core的RDD访问带有数据的文件.

That's mainly for Spark Core's RDD to access files with data.

其他存储系统(如Cassandra，s3，HBase，SQL数据库，NoSQL数据库)如何?我们是否需要下载任何库才能连接到其他存储系统.

how about other storage systems like Cassandra, s3, HBase, SQL Databases, NoSQL databases? Do we need to download any libraries to connect to other storage systems.

在上述存储系统中，为Apache Hadoop 2.7及更高版本预构建"捆绑包部分覆盖了S3(但是您必须专门为S3添加其他jar).

Out of the mentioned storage systems, S3 is covered partially by the "pre-built for Apache Hadoop 2.7 and later" bundle (but you have to add additional jars for S3 specifically).

主要用于Spark SQL的数据集API.

That's mostly for Spark SQL's Dataset API.

Cassandra，HBase等具有自己的Spark连接器，不包含在内.参见 DataStax Spark Cassandra Connector 和

Cassandra, HBase, etc. have their own Spark connectors and are not included. See DataStax Spark Cassandra Connector and Apache HBase Connector

这篇关于“针对Apache Hadoop 2.7及更高版本预构建"的内容是什么?意思是?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！