无法使用WebHDFS列出Data Lake中的文件夹内容（服务器无法验证请求） [英] Cannot list folder contents in Data Lake using WebHDFS (Server failed to authenticate the request)
问题描述
你好，
我在使用WebHDFS REST调用来列出/获取Azure Data Lake存储上的文件夹内容时遇到了一些麻烦。
我基本上遵循指南"通过WEBHDFS访问ADL商店";
https://social.msdn.microsoft.com/Forums/azure/ en-US / cd7dee04-19a4-4304-8e2c-20c70bc8a5b9 / access-to-adl-store-by-webhdfs
当我尝试REST呼叫列表时文件夹内容（op = LISTSTATUS）我收到错误回复：
" < Code> InvalidAuthenticationInfo< / Code>< Message>身份验证信息的格式不正确。检查授权标题的值。
RequestId：3e5e0c23-401a-0092-2950-f4cb03000000
时间：2019-04-16T12：33：56.2279004Z< /消息> "
$
以下是卷曲电话给出上述错误：
curl -X GET -H" x-ms-version：2018-11-09" -H"授权：承载< ACCESS-TOKEN>" https://contentcloudtestdatalake.file.core.windows.net/webhdfs/v1/?op=LISTSTATUS
你有什么提示我有什么做错了，或者我如何调试并找出"授权标题"中可能缺少的内容。 ？
Hello,
I'm having some trouble using WebHDFS REST calls to list/get folder contents on our Azure Data Lake storage.
I have basically followed the guide "Access to ADL Store by WEBHDFS";
https://social.msdn.microsoft.com/Forums/azure/en-US/cd7dee04-19a4-4304-8e2c-20c70bc8a5b9/access-to-adl-store-by-webhdfs
When i try the REST-call to list folder contents (op=LISTSTATUS) i get an error in return:
"<Code>InvalidAuthenticationInfo</Code><Message>Authentication information is not given in the correct format. Check the value of Authorization header.
RequestId:3e5e0c23-401a-0092-2950-f4cb03000000
Time:2019-04-16T12:33:56.2279004Z</Message>"
Below is the curl-call that gives the error above:
curl -X GET -H "x-ms-version: 2018-11-09" -H "Authorization: Bearer <ACCESS-TOKEN>" https://contentcloudtestdatalake.file.core.windows.net/webhdfs/v1/?op=LISTSTATUS
Do you have any tips on what i have done wrong, or how i can debug and find out what might be missing in the "authorization header" ?
推荐答案
Hi Nils，
Hi Nils,
您使用的是Azure Data Lake Gen2吗？在这种情况下，请查看
上的建议答案
this thread 。我已经发布了一个使用ADLS Gen2 REST API的综合分步指南。
Are you using Azure Data Lake Gen2 ? In that case, please have a look at the proposed answer on this thread. I have posted a comprehensive step-by-step guide to use ADLS Gen2 REST APIs.
除了上面的帖子，列出文件夹的内容，
In addition to the above post, to list the contents of a folder,
要从根目录列出所有文件，请在替换变量值后运行以下命令：
curl -H" x-ms-版本：2018-11-09" -H"授权：承载
To list all files recursively from the root, run the following commands after replacing variable values:
curl -H "x-ms-version: 2018-11-09" -H "Authorization: Bearer
ACCESS_TOKEN" " https：//
ACCESS_TOKEN" "https://
STORAGE_ACCOUNT_NAME.dfs.core.windows.net/mydata?resource=filesystem&recursive=true"
来自服务器将类似于此JSON片段：
{
" paths"：[
{
" isDirectory"：" true"，
" name"：" data"，
" permissions"：" rwxr-x ---"
}，
{
" contentLength"：" 44"，
" name"：" data / file1"，$
" permissions"：" rw-r -----"
}，
{
$
" contentLength"：" 0"，
" name"：" data / file2"，
" permissions"：" rwxrwx --- +"
}
] b
$
}
要仅列出单个目录中的文件，请运行以下命令：
$
curl -H" x-ms-version：2018-11-09" ; -H"授权：持票人
STORAGE_ACCOUNT_NAME.dfs.core.windows.net/mydata?resource=filesystem&recursive=true"
The response from the server will be similar to this JSON fragment:
{
"paths": [
{
"isDirectory": "true",
"name": "data",
"permissions": "rwxr-x---"
},
{
"contentLength": "44",
"name": "data/file1",
"permissions": "rw-r-----"
},
{
"contentLength": "0",
"name": "data/file2",
"permissions": "rwxrwx---+"
}
]
}
To list just the files in a single directory, run this command:
curl -H "x-ms-version: 2018-11-09" -H "Authorization: Bearer
这篇关于无法使用WebHDFS列出Data Lake中的文件夹内容（服务器无法验证请求）的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
