如何使用Hadoop FileSystem API从本地macine（intelliJ）访问ADLS Gen2中的文件 [英] How to access files in ADLS Gen2 from local macine (intelliJ) using Hadoop FileSystem API

问题描述

 

我正在尝试使用Hadoop文件系统API从java访问ADLS Gen 2容器。
" fs.azure.account.oauth.provider.type" 在core-site.xml中设置为
" com.microsoft.azure.storage.oauth2.TokenServiceBasedAccessTokenProvider"
这是我在互联网上找不到的。经过很多痛苦，我发现课程出现在罐子里  （adls2-oauth2-token-provider-1.0.jar，adls2 -auth2-token-provider.jar）在"hadoop-hdfs / libs /"下。在我的HDInsight集群上，我创建了
这个天蓝色的门户网站，并在创建它的时候分配了第二代存储帐户。

I'm trying to access an ADLS Gen 2 container from java using Hadoop File System API. The "fs.azure.account.oauth.provider.type" in core-site.xml is set to "com.microsoft.azure.storage.oauth2.TokenServiceBasedAccessTokenProvider" which I could not find on the internet.. After a lot of pain, I found that the class is present in jars  (adls2-oauth2-token-provider-1.0.jar, adls2-oauth2-token-provider.jar) under "hadoop-hdfs/libs/" on my HDInsight cluster which I created form the azure portal and to which I assigned a Gen 2 storage account while creating it.

在我的类路径中添加这些jar后，核心站点。，我仍然无法访问它，因为属性值为
" fs.azure.datalake.gen2.token.provider.service.urls"的主机名。不是公共主机名。因此出现错误" java.net.UnknownHostException"

After I added these jars in my classpath along with the core-site., I'm still not able to access it because the hostnames in the value of property "fs.azure.datalake.gen2.token.provider.service.urls" are not public hostnames. So the error comes up "java.net.UnknownHostException"

推荐答案

您好Amruth，

Hi Amruth,

ADLS Gen 2是一个基于存储帐户构建的Analytics的专用存储帐户。因此，它使用名为Azure Blob文件系统驱动程序的驱动程序与hadoop一起使用。 

ADLS Gen 2 is a specialized storage account with Analytics built on top of a storage account. Hence, it works with hadoop using a driver called Azure Blob File System driver. 

要了解更多信息，请参阅
此文档。

To read more about it, please refer this doc.

希望这会有所帮助。

这篇关于如何使用Hadoop FileSystem API从本地macine（intelliJ）访问ADLS Gen2中的文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！