使用bash访问的Azure Blob存储，卷曲 [英] Accessing Azure blob storage using bash, curl

问题描述

我试图从使用REST API一个bash脚本中使用的Azure Blob存储服务。我知道这是可能做到这一点使用各种其他工具或语言，但我希望做一个bash脚本。

下面的脚本是试图列出斑点在Azure存储容器。

该脚本会导致身份验证错误。签署字符串和头基于REST的API（参考）文档。我怀疑问题可能出在杂耍签名过程的各个部分。

有没有人成功地使用bash和卷曲访问像Azure的或其他供应商？

云存储资源

＃！/斌/庆典＃列表在Azure存储容器的斑点。回声用法：$ {0 ## * /}＆LT;存储帐户名称＆gt;＆LT;容器名称＆gt;＆LT;获得密钥＆gt;中STORAG​​E_ACCOUNT =$ 1
CONTAINER_NAME =$ 2
access_key =$ 3blob_store_url =blob.core.windows.net
授权=SharedKeyREQUEST_METHOD =GET
REQUEST_DATE = $（TZ = GMT日期+％A％D％H％Y％H：％M：％S％Z）
storage_service_version =2011-08-18＃HTTP请求头
x_ms_date_h =X-MS-日期：$ REQUEST_DATE
x_ms_version_h =X-MS-版本：$ storage_service_version＃构建的署名字符串
canonicalized_headers =$ {} x_ms_date_h \\ n $ {} x_ms_version_h
canonicalized_resource =/ $ {} STORAG​​E_ACCOUNT / $ {} CONTAINER_NAMEstring_to_sign=\"${request_method}\
\
\
\
\
\
\
\
\
\
\
\
${canonicalized_headers}\
${canonicalized_resource}\
comp:list\
restype:container\"德＃code为Base64 EN codeD快捷键，转换为十六进制。
德coded_hex_key =$（回声-n $ access_key | -d的base64 -w0 | XXD -p -c256）＃创建Authorization头的HMAC签名
签名= $（回声-n$ string_to_sign| OpenSSL的DGST -sha256 -mac HMAC -macopthexkey：$德coded_hex_key| sed中的/^.*= ​​//'|的base64 -w0）authorization_header =授权：授权$ $ STORAG​​E_ACCOUNT：$签名卷曲\\
  -H$ x_ms_date_h\\
  -H$ x_ms_version_h\\
  -H$ authorization_header\\
  https：//开头$ {STORAG​​E_ACCOUNT} $ {blob_store_url} / $ {} CONTAINER_NAME = restype容器和放大器;可比=名单？


更新 - 存储服务错误，并且生成的脚本相应的字符串签署

以下是存储服务返回 AuthenticationFailed 错误。

＆LT;？XML版本=1.0编码=UTF-8＆GT？;
＆LT;错误＆GT;
  ＆LT; code基AuthenticationFailed＆LT; / code＆GT;
  ＆LT;消息＆GT;服务器无法验证请求。确保授权头的值是正确形成，包括签名。
请求ID：27e6337e-52f3-4e85-98c7-2fabaacd9ebc
时间：2013-11-21T22：10：11.7029042Z＆LT; /信息＆GT;
  ＆LT; AuthenticationErrorDetail＆gt;将MAC签名在HTTP请求中发现
OGYxYjk1MTFkYmNkMCgzN2YzODQwNzcyNiIyYTQxZDg0OWFjNGJiZDlmNWY5YzM1ZWQzMWViMGFjYTAyZDY4NAo ='
是不一样的任何计算签名。使用下面的字符串签署服务器：
'得到X-MS-日期：星期四，2013年11月21日22点10分十一秒GMT
X-MS-版本：2011-08-18
/ storage_account_name / storage_container
补偿：列表
restype：容器
  ＆LT; / AuthenticationErrorDetail＆GT;
＆LT; /错误＆GT;


接下来是 string_to_sign 脚本生成。

GET \\ n \\ n \\ n \\ n \\ n \\ n \\ n \\ n \\ n \\ n \\ n \\ NX -ms-日期：星期四，2013年11月21日22时十分十一秒GMT \\ NX-MS-版本：2011-08-18 \\ N / storage_account_name / storage_container \\ NCOMP：列表\\ nrestype：容器


解决方案

我能得到它的工作。
有两件事情不对的code，第一，帕特里克园指出，被取代回声-n 与的printf 。第二次是取代与OpenSSL中 - 二进制选项 SED 魔法。

比较原始的：

 签名= $（回声-n$ string_to_sign| OpenSSL的DGST -sha256 -mac HMAC -macopthexkey：$德coded_hex_key-binary | sed的'S / ^ * = //'|的base64 -w0）
 

与固定的：

 签名= $（printf的$ string_to_sign| OpenSSL的DGST -sha256 -mac HMAC -macopthexkey：$德coded_hex_key-binary | BASE64 -w0）
 

是必须的回声改变，因为回声-n 将无法转换 \\ n 成实际换行。

是必须的 - 二进制的变化，因为即使你剥离坏的部分，OpenSSL的仍然在输出ASCII-CN codeD-十六进制的签名，不是二进制。它被传递到的base64 所以后，得到的结果是十六进制再presentation的B64 EN codeD版本，而不是原始值。

I am attempting to use the Azure blob storage service from a bash script using the REST API. I know it is possible to accomplish this using various other tools or languages, however I'd like to do it as a bash script.

The script below is an attempt to list the blobs in an Azure storage container.

This script results in an authentication error. The signing string and headers look correct based on the REST API (reference) documentation. I suspect the problem may be in juggling the various parts of the signing process.

Has anyone successfully used bash and curl to access cloud storage resources like Azure or other providers?

#!/bin/bash

# List the blobs in an Azure storage container.

echo "usage: ${0##*/} <storage-account-name> <container-name> <access-key>"

storage_account="$1"
container_name="$2"
access_key="$3"

blob_store_url="blob.core.windows.net"
authorization="SharedKey"

request_method="GET"
request_date=$(TZ=GMT date "+%a, %d %h %Y %H:%M:%S %Z")
storage_service_version="2011-08-18"

# HTTP Request headers
x_ms_date_h="x-ms-date:$request_date"
x_ms_version_h="x-ms-version:$storage_service_version"

# Build the signature string
canonicalized_headers="${x_ms_date_h}\n${x_ms_version_h}"
canonicalized_resource="/${storage_account}/${container_name}"

string_to_sign="${request_method}\n\n\n\n\n\n\n\n\n\n\n\n${canonicalized_headers}\n${canonicalized_resource}\ncomp:list\nrestype:container"

# Decode the Base64 encoded access key, convert to Hex.
decoded_hex_key="$(echo -n $access_key | base64 -d -w0 | xxd -p -c256)"

# Create the HMAC signature for the Authorization header
signature=$(echo -n "$string_to_sign" | openssl dgst -sha256 -mac HMAC -macopt "hexkey:$decoded_hex_key" | sed 's/^.*= //' | base64 -w0)

authorization_header="Authorization: $authorization $storage_account:$signature"

curl \
  -H "$x_ms_date_h" \
  -H "$x_ms_version_h" \
  -H "$authorization_header" \
  "https://${storage_account}.${blob_store_url}/${container_name}?restype=container&comp=list"

Update - The storage service error and the corresponding signing string that the script generated.

Following is what the storage service returns for the AuthenticationFailed error.

<?xml version="1.0" encoding="utf-8"?>
<Error>
  <Code>AuthenticationFailed</Code>
  <Message>Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
RequestId:27e6337e-52f3-4e85-98c7-2fabaacd9ebc
Time:2013-11-21T22:10:11.7029042Z</Message>
  <AuthenticationErrorDetail>The MAC signature found in the HTTP request
'OGYxYjk1MTFkYmNkMCgzN2YzODQwNzcyNiIyYTQxZDg0OWFjNGJiZDlmNWY5YzM1ZWQzMWViMGFjYTAyZDY4NAo='
is not the same as any computed signature. Server used following string to sign:
'GET

x-ms-date:Thu, 21 Nov 2013 22:10:11 GMT
x-ms-version:2011-08-18
/storage_account_name/storage_container
comp:list
restype:container'
  </AuthenticationErrorDetail>
</Error>

Next is the string_to_sign that the script generates.

GET\n\n\n\n\n\n\n\n\n\n\n\nx-ms-date:Thu, 21 Nov 2013 22:10:11 GMT\nx-ms-version:2011-08-18\n/storage_account_name/storage_container\ncomp:list\nrestype:container

解决方案

I was able to get it working. There were two things wrong with this code, the first, as Patrick Park noted, was replacing the echo -n with printf. The second was replacing the sed magic with the -binary option on openssl.

Compare the original:

signature=$(echo -n "$string_to_sign" | openssl dgst -sha256 -mac HMAC -macopt "hexkey:$decoded_hex_key" -binary | sed 's/^.*= //' | base64 -w0)

with the fixed:

signature=$(printf "$string_to_sign" | openssl dgst -sha256 -mac HMAC -macopt "hexkey:$decoded_hex_key" -binary |  base64 -w0)

The echo change is needed because echo -n will not convert the \n into actual newlines.

The -binary change is needed because even though you are stripping off the bad part, openssl was still outputting the signature in ascii-encoded-hex, not in binary. So after it was passed to base64, the result was the b64 encoded version of the hex representation, instead of the raw value.

这篇关于使用bash访问的Azure Blob存储，卷曲的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！