使用 bash、curl 访问 Azure blob 存储 [英] Accessing Azure blob storage using bash, curl

问题描述

我正在尝试通过使用 REST API 的 bash 脚本使用 Azure blob 存储服务.我知道可以使用其他各种工具或语言来完成此操作，但我想将其作为 bash 脚本来完成.

I am attempting to use the Azure blob storage service from a bash script using the REST API. I know it is possible to accomplish this using various other tools or languages, however I'd like to do it as a bash script.

下面的脚本试图列出 Azure 存储容器中的 blob.

The script below is an attempt to list the blobs in an Azure storage container.

此脚本导致身份验证错误.基于 REST API(参考) 文档.我怀疑问题可能在于处理签名过程的各个部分.

This script results in an authentication error. The signing string and headers look correct based on the REST API (reference) documentation. I suspect the problem may be in juggling the various parts of the signing process.

有没有人成功使用 bash 和 curl 访问 Azure 或其他提供商等云存储资源?

Has anyone successfully used bash and curl to access cloud storage resources like Azure or other providers?

#!/bin/bash

# List the blobs in an Azure storage container.

echo "usage: ${0##*/} <storage-account-name> <container-name> <access-key>"

storage_account="$1"
container_name="$2"
access_key="$3"

blob_store_url="blob.core.windows.net"
authorization="SharedKey"

request_method="GET"
request_date=$(TZ=GMT date "+%a, %d %h %Y %H:%M:%S %Z")
storage_service_version="2011-08-18"

# HTTP Request headers
x_ms_date_h="x-ms-date:$request_date"
x_ms_version_h="x-ms-version:$storage_service_version"

# Build the signature string
canonicalized_headers="${x_ms_date_h}
${x_ms_version_h}"
canonicalized_resource="/${storage_account}/${container_name}"

string_to_sign="${request_method}











${canonicalized_headers}
${canonicalized_resource}
comp:list
restype:container"

# Decode the Base64 encoded access key, convert to Hex.
decoded_hex_key="$(echo -n $access_key | base64 -d -w0 | xxd -p -c256)"

# Create the HMAC signature for the Authorization header
signature=$(echo -n "$string_to_sign" | openssl dgst -sha256 -mac HMAC -macopt "hexkey:$decoded_hex_key" | sed 's/^.*= //' | base64 -w0)

authorization_header="Authorization: $authorization $storage_account:$signature"

curl 
  -H "$x_ms_date_h" 
  -H "$x_ms_version_h" 
  -H "$authorization_header" 
  "https://${storage_account}.${blob_store_url}/${container_name}?restype=container&comp=list"

更新 - 存储服务错误和脚本生成的相应签名字符串.

Update - The storage service error and the corresponding signing string that the script generated.

以下是存储服务针对 AuthenticationFailed 错误返回的内容.

Following is what the storage service returns for the AuthenticationFailed error.

<?xml version="1.0" encoding="utf-8"?>
<Error>
  <Code>AuthenticationFailed</Code>
  <Message>Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
RequestId:27e6337e-52f3-4e85-98c7-2fabaacd9ebc
Time:2013-11-21T22:10:11.7029042Z</Message>
  <AuthenticationErrorDetail>The MAC signature found in the HTTP request
'OGYxYjk1MTFkYmNkMCgzN2YzODQwNzcyNiIyYTQxZDg0OWFjNGJiZDlmNWY5YzM1ZWQzMWViMGFjYTAyZDY4NAo='
is not the same as any computed signature. Server used following string to sign:
'GET

x-ms-date:Thu, 21 Nov 2013 22:10:11 GMT
x-ms-version:2011-08-18
/storage_account_name/storage_container
comp:list
restype:container'
  </AuthenticationErrorDetail>
</Error>

接下来是脚本生成的string_to_sign.

GET











x-ms-date:Thu, 21 Nov 2013 22:10:11 GMT
x-ms-version:2011-08-18
/storage_account_name/storage_container
comp:list
restype:container

推荐答案

我能够让它工作.这段代码有两个问题，第一，正如帕特里克帕克指出的那样，用 printf 替换了 echo -n.第二个是用 openssl 上的 -binary 选项替换 sed 魔法.

I was able to get it working. There were two things wrong with this code, the first, as Patrick Park noted, was replacing the echo -n with printf. The second was replacing the sed magic with the -binary option on openssl.

对比原文:

signature=$(echo -n "$string_to_sign" | openssl dgst -sha256 -mac HMAC -macopt "hexkey:$decoded_hex_key" -binary | sed 's/^.*= //' | base64 -w0)

固定:

signature=$(printf "$string_to_sign" | openssl dgst -sha256 -mac HMAC -macopt "hexkey:$decoded_hex_key" -binary |  base64 -w0)

需要更改 echo，因为 echo -n 不会将 转换为实际的换行符.

The echo change is needed because echo -n will not convert the into actual newlines.

-binary 更改是必需的，因为即使您剥离了坏部分，openssl 仍然以 ascii-encoded-hex 格式输出签名，而不是二进制格式.因此，在将其传递给 base64 之后，结果是十六进制表示的 b64 编码版本，而不是原始值.

The -binary change is needed because even though you are stripping off the bad part, openssl was still outputting the signature in ascii-encoded-hex, not in binary. So after it was passed to base64, the result was the b64 encoded version of the hex representation, instead of the raw value.

这篇关于使用 bash、curl 访问 Azure blob 存储的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！