用于访问Azure Data Lake Store的Python代码 [英] Python code to access Azure Data Lake Store

问题描述

我正在查看Microsoft文档此处和

I 'm looking at Microsoft Documentation here and here, I have created Web App in Azure Active Directory to access the Data Lake Store

在Web应用程序中，我具有对象ID ，应用 ID和密钥

From the Web App I have Object ID, Application ID and Key

查看文档，我会看到以下内容:

looking at the documentations I see this:

adlCreds = lib.auth(tenant_id = 'FILL-IN-HERE', client_secret = 'FILL-IN-HERE', client_id = 'FILL-IN-HERE', resource = 'https://datalake.azure.net/')

如何使用它来验证我的代码并在Data Lake Store上运行操作?

how to use it to authenticate my code and run operation on Data Lake Store?

这是我的完整测试代码:

here is my full test code:

## Use this for Azure AD authentication
from msrestazure.azure_active_directory import AADTokenCredentials

## Required for Azure Data Lake Store account management
from azure.mgmt.datalake.store import DataLakeStoreAccountManagementClient
from azure.mgmt.datalake.store.models import DataLakeStoreAccount

## Required for Azure Data Lake Store filesystem management
from azure.datalake.store import core, lib, multithread

# Common Azure imports
import adal
from azure.mgmt.resource.resources import ResourceManagementClient
from azure.mgmt.resource.resources.models import ResourceGroup

## Use these as needed for your application
import logging, getpass, pprint, uuid, time


## Declare variables
subscriptionId = 'FILL-IN-HERE'
adlsAccountName = 'FILL-IN-HERE'

tenant_id = 'FILL-IN-HERE'
client_secret = 'FILL-IN-HERE'
client_id = 'FILL-IN-HERE'


## adlCreds = lib.auth(tenant_id = 'FILL-IN-HERE', client_secret = 'FILL-IN-HERE', client_id = 'FILL-IN-HERE', resource = 'https://datalake.azure.net/')
from azure.common.credentials import ServicePrincipalCredentials
adlCreds = lib.auth(tenant_id, client_secret, client_id, resource = 'https://datalake.azure.net/')


## Create a filesystem client object
adlsFileSystemClient = core.AzureDLFileSystem(adlCreds, store_name=adlsAccountName)

## Create a directory
adlsFileSystemClient.mkdir('/mysampledirectory')

当我尝试查看代码时，我得到了错误:

when I try to ru the code I get error:

[正在运行] python"c:.... \ dls.py"追溯(最近一次通话):在第38行中输入文件"c:.... \ dls.py"adlCreds = lib.auth(tenant_id，client_secret，client_id，resource =' https://datalake.azure.net/')文件"C:\ Python36 \ lib \ site-packages \ azure \ datalake \ store \ lib.py"，行130，身份验证密码，client_id)文件"C:\ Python36 \ lib \ site-packages \ adal \ authentication_context.py"，第145行，在acquire_token_with_username_password中返回self._acquire_token(token_func)_acquire_token中的第109行的文件"C:\ Python36 \ lib \ site-packages \ adal \ authentication_context.py"返回token_func()token_func中的文件"C:\ Python36 \ lib \ site-packages \ adal \ authentication_context.py"，第143行返回token_request.get_token_with_username_password(用户名，密码)get_token_with_username_password中的第280行的文件"C:\ Python36 \ lib \ site-packages \ adal \ token_request.py"self._user_realm.discover()发现中的文件"C:\ Python36 \ lib \ site-packages \ adal \ user_realm.py"，第152行引发AdalError(return_error_string，error_response)adal.adal_error.AdalError:用户领域发现请求返回了http错误:404和服务器响应:

[Running] python "c:....\dls.py" Traceback (most recent call last): File "c:....\dls.py", line 38, in adlCreds = lib.auth(tenant_id, client_secret, client_id, resource = 'https://datalake.azure.net/') File "C:\Python36\lib\site-packages\azure\datalake\store\lib.py", line 130, in auth password, client_id) File "C:\Python36\lib\site-packages\adal\authentication_context.py", line 145, in acquire_token_with_username_password return self._acquire_token(token_func) File "C:\Python36\lib\site-packages\adal\authentication_context.py", line 109, in _acquire_token return token_func(self) File "C:\Python36\lib\site-packages\adal\authentication_context.py", line 143, in token_func return token_request.get_token_with_username_password(username, password) File "C:\Python36\lib\site-packages\adal\token_request.py", line 280, in get_token_with_username_password self._user_realm.discover() File "C:\Python36\lib\site-packages\adal\user_realm.py", line 152, in discover raise AdalError(return_error_string, error_response) adal.adal_error.AdalError: User Realm Discovery request returned http error: 404 and server response:

404-找不到文件或目录.

404 - File or directory not found.

[Done]在1.216秒内以代码= 1退出

[Done] exited with code=1 in 1.216 seconds

推荐答案

有两种不同的身份验证方法.第一个是交互式的，适合最终用户.它甚至可以与多因素身份验证一起使用.这是您的操作方式.您需要互动才能登录.

There are two different ways of authenticating. The first one is interactive which is suitable for end users. It even works with multi factor authentication. Here is how you do it. You need to be interactive in order to log on.

from azure.datalake.store import core, lib, multithread
token = lib.auth()

第二种方法是在Azure Active Directory中使用服务主体身份.有关如何设置Azure AD应用程序，检索客户端ID和机密以及使用SPI配置访问的循序渐进教程，可以在这里找到:

The second method is to use service principal identities in Azure Active directory. A step by step tutorial for setting up an Azure AD application, retrieving the client id and secret and configuring access using the SPI is available here: https://docs.microsoft.com/en-us/azure/data-lake-store/data-lake-store-service-to-service-authenticate-using-active-directory#create-an-active-directory-application

from azure.common.credentials import ServicePrincipalCredentials
token = lib.auth(tenant_id = '<your azure tenant id>', client_secret = '<your client secret>', client_id = '<your client id>')

这是一篇博客文章，显示如何通过熊猫和Jupyter访问它.它还逐步介绍了如何获取身份验证令牌. https://medium.com/azure-data-lake/using-jupyter-notebooks-and-pandas-with-azure-data-lake-store-48737fbad305

Here is blog post that shows how to access it through pandas and Jupyter. It also has a step by step on how to get the authentication token. https://medium.com/azure-data-lake/using-jupyter-notebooks-and-pandas-with-azure-data-lake-store-48737fbad305

这篇关于用于访问Azure Data Lake Store的Python代码的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！