如何在不安装AWS开发工具包的情况下通过Powershell从S3下载文件? [英] How can I download a file from S3 via Powershell without installing AWS SDK?

问题描述

我想使用Windows Powershell从我的AWS S3存储桶下载文件.我无法安装任何AWS软件，需要创建一个API才能访问AWS S3中的文件.我使用Postman来测试该文件是否可访问并且成功.

鉴于此成功，我尝试遵循AWS指南，该指南指出我需要创建以下内容:

• 创建规范请求.
• 使用规范请求和其他元数据创建字符串进行签名.
• 从您的AWS秘密访问密钥中获取签名密钥.然后使用签名密钥以及上一步中的字符串，以创建一个签名.
• 将结果签名添加到HTTP请求的标头中或作为查询字符串参数.

我所看到的最接近的例子是来自

解决方案

存在一些错误，特别是您如何计算签名，建立时间戳记以及所看到的错误是因为参数设置不正确被传递.

以下是纠正这些问题的版本:

  $ method ='GET'$ service ='s3'$ bucket ="SAMPLES3BUCKETNAME"$ key ='test.xlsx'$ region ='ap-southeast-1'$ host1 = $ bucket +'.s3-'+ $ region +'.amazonaws.com'$ access_key ='SAMPLEACCESSKEY'$ secret_key ='SAMPLESECRETKEY'函数HmacSHA256($ message，$ secret){$ hmacsha =新对象System.Security.Cryptography.HMACSHA256$ hmacsha.key = $秘密$ signature = $ hmacsha.ComputeHash([Text.Encoding] :: ASCII.GetBytes($ message))返回$ sign}函数getSignatureKey($ key，$ dateStamp，$ regionName，$ serviceName){$ kSecret = [Text.Encoding] :: UTF8.GetBytes(("AWS4" + $ key).toCharArray())$ kDate = HmacSHA256 $ dateStamp $ kSecret;$ kRegion = HmacSHA256 $ regionName $ kDate;$ kService = HmacSHA256 $ serviceName $ kRegion;$ kSigning = HmacSHA256"aws4_request" $ kService;返回$ k}函数hash($ request){$ hasher = [System.Security.Cryptography.SHA256] :: Create()$ content = [Text.Encoding] :: UTF8.GetBytes($ request)$ bytes = $ hasher.ComputeHash($ content)返回($ bytes | ForEach-Object ToString x2)-join''}$ now = [DateTime] :: UtcNow$ amz_date = $ now.ToString('yyyyMMddTHHmmssZ')$ datestamp = $ now.ToString('yyyyMMdd')$ signed_headers ='主机'$ credential_scope = $ datestamp +'/'+ $ region +'/'+ $ service +'/'+'aws4_request'$ canonical_querystring ='X-Amz-Algorithm = AWS4-HMAC-SHA256'$ canonical_querystring + ='& X-Amz-Credential ='+ [uri] :: EscapeDataString(($ access_key +'/'+ $ credential_scope))$ canonical_querystring + ='& X-Amz-Date ='+ $ amz_date$ canonical_querystring + ='& X-Amz-Expires = 86400'$ canonical_querystring + ='& X-Amz-SignedHeaders ='+ $ signed_headers$ canonical_headers ='主机:'+ $ host1 +`n"$ canonical_request = $ method +`n"$ canonical_request + ="/" + $ key +`n"$ canonical_request + = $ canonical_querystring +`n"$ canonical_request + = $ canonical_headers +`n"$ canonical_request + = $ signed_headers +`n"$ canonical_request + =未签名的负载"$ algorithm ='AWS4-HMAC-SHA256'$ canonical_request_hash =哈希-request $ canonical_request$ string_to_sign = $算法+`n"$ string_to_sign + = $ amz_date +`n"$ string_to_sign + = $ credential_scope +`n"$ string_to_sign + = $ canonical_request_hash$ signing_key = getSignatureKey $ secret_key $ datestamp $ region $ service$ signature = HmacSHA256-秘密$ signing_key-消息$ string_to_sign$ signature =($ signature | ForEach-Object ToString x2)-join''$ canonical_querystring + ='& X-Amz-Signature ='+ $ signature$ request_url ="http://" + $ host1 +"/" + $ key +?"+ $ canonical_querystring写主机$ request_url 

I want to download a file from my AWS S3 bucket using Windows Powershell. I cannot install any AWS software and need to create an API to be able to access a file in AWS S3. I used Postman for testing that the file is accessible and it was successful.

Given this success I tried following AWS' guide which says that I need to create the following:

• Create a canonical request.
• Use the canonical request and additional metadata to create a string for signing.
• Derive a signing key from your AWS secret access key. Then use the signing key, and the string from the previous step, to create a signature.
• Add the resulting signature to the HTTP request in a header or as a query string parameter.

The closest I've seen is this example from https://forums.aws.amazon.com/thread.jspa?threadID=251722 by Abhaya however it is also unresolved. (The payload hash in this example is the payload hash for blank). I have gone through several AWS guides but these are very confusing when trying to apply them to powershell. https://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html

The url the code below generates a url that looks correct:http://SAMPLEBUCKETNAME HERE.s3-ap-southeast-1.amazonaws.com/test.xlsx?&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=%2F20190907%2Fap-southeast-1%2Fs3%2Faws4_request&X-Amz-Date=20190907T1644136560000Z&X-Amz-E0&X-Amz-SignedHeaders=host&X-Amz-Signature=HASH HERE

$method = 'GET'
$service = 's3'
$host1 = 'SAMPLES3BUCKETNAME.s3-ap-southeast-1.amazonaws.com'
$region = 'ap-southeast-1'
$endpoint = 'http://SAMPLES3BUCKETNAME.s3-ap-southeast-1.amazonaws.com/test.xlsx'

function HmacSHA256($message, $secret){
   <#$hmacsha = New-Object System.Security.Cryptography.HMACSHA256
    $hmacsha.key = [Text.Encoding]::UTF8.GetBytes($secret)
    #$hmacsha.key = $secret
    $signature = $hmacsha.ComputeHash([Text.Encoding]::UTF8.GetBytes($message))
    $signature = [Convert]::ToBase64String($signature)
   #>
    $hmacsha = New-Object System.Security.Cryptography.HMACSHA256

    $hmacsha.Key = @($secret -split '(?<=\G..)(?=.)'|ForEach-Object {[byte]::Parse($_,'HexNumber')})

    $sign = [BitConverter]::ToString($hmacsha.ComputeHash([Text.Encoding]::UTF8.GetBytes($message))).Replace('-','').ToLower()

    return $sign
}

function getSignatureKey($key, $dateStamp, $regionName, $serviceName)
{
    $kSecret = [Text.Encoding]::UTF8.GetBytes(("AWS4" + $key).toCharArray())
    $kDate = HmacSHA256 $dateStamp $kSecret;
    $kRegion = HmacSHA256 $regionName $kDate ;
    $kService = HmacSHA256 $serviceName $kRegion ;
    $kSigning = HmacSHA256 "aws4_request" $kService ;

    return $kSigning;
}


$access_key = 'SAMPLEACCESSKEY'
$secret_key = 'SAMPLESECRETKEY'

$amz_date = [DateTime]::UtcNow.ToString('yyyyMMddTHHmmssfffffffZ')
$datestamp = [DateTime]::UtcNow.ToString('yyyyMMdd')

$canonical_uri = '/'
$canonical_headers = 'host:' + $host1 + "`n"
$signed_headers = 'host'

$algorithm = 'AWS4-HMAC-SHA256'
$credential_scope = $datestamp + '/' + $region + '/' + $service + '/' + 'aws4_request'

$canonical_querystring = ''
$canonical_querystring += '&X-Amz-Algorithm=AWS4-HMAC-SHA256'
$canonical_querystring += '&X-Amz-Credential=' + [uri]::EscapeDataString(($access_key + '/' + $credential_scope))
$canonical_querystring += '&X-Amz-Date=' + $amz_date
$canonical_querystring += '&X-Amz-Expires=86400'
$canonical_querystring += '&X-Amz-SignedHeaders=' + $signed_headers


$payload_hash = 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'

$canonical_request1 = $method + "`n" +$canonical_uri + "`n" + $canonical_querystring + "`n" + $canonical_headers + "`n" + $signed_headers + "`n" + $payload_hash
Write-Host $canonical_request1

function hash($request) {

$hasher = [System.Security.Cryptography.SHA256]::Create()
$content = [Text.Encoding]::UTF8.GetBytes($request)
$hash = [System.Convert]::ToBase64String($hasher.ComputeHash($content))
return $hash

}

$canonical_request = hash -request $canonical_request1
$string_to_sign = $algorithm + "`n" +  $amz_date + "`n" +  $credential_scope + "`n" +  $canonical_request


$signing_key = getSignatureKey $secret_key $datestamp $region $service
$signature =  HmacSHA256 -secret $signing_key -message $string_to_sign

$canonical_querystring += '&X-Amz-Signature=' + $signature

$request_url = $endpoint + "?" + $canonical_querystring
$request_url

I get the following error when I try accessing the url.

解决方案

There were a few errors, notably how you were computing the signature, building the timestamp, and the error that you were seeing is because the parameters weren't properly being passed along.

Here's a version that corrects those issues:

$method = 'GET'
$service = 's3'
$bucket = "SAMPLES3BUCKETNAME"
$key = 'test.xlsx'
$region = 'ap-southeast-1'
$host1 = $bucket + '.s3-' + $region + '.amazonaws.com'
$access_key = 'SAMPLEACCESSKEY'
$secret_key = 'SAMPLESECRETKEY'

function HmacSHA256($message, $secret)
{
    $hmacsha = New-Object System.Security.Cryptography.HMACSHA256
    $hmacsha.key = $secret
    $signature = $hmacsha.ComputeHash([Text.Encoding]::ASCII.GetBytes($message))

    return $signature
}

function getSignatureKey($key, $dateStamp, $regionName, $serviceName)
{
    $kSecret = [Text.Encoding]::UTF8.GetBytes(("AWS4" + $key).toCharArray())
    $kDate = HmacSHA256 $dateStamp $kSecret;
    $kRegion = HmacSHA256 $regionName $kDate;
    $kService = HmacSHA256 $serviceName $kRegion;
    $kSigning = HmacSHA256 "aws4_request" $kService;

    return $kSigning
}

function hash($request)
{
    $hasher = [System.Security.Cryptography.SHA256]::Create()
    $content = [Text.Encoding]::UTF8.GetBytes($request)
    $bytes = $hasher.ComputeHash($content)

    return ($bytes|ForEach-Object ToString x2) -join ''
}

$now = [DateTime]::UtcNow
$amz_date = $now.ToString('yyyyMMddTHHmmssZ')
$datestamp = $now.ToString('yyyyMMdd')

$signed_headers = 'host'
$credential_scope = $datestamp + '/' + $region + '/' + $service + '/' + 'aws4_request'

$canonical_querystring = 'X-Amz-Algorithm=AWS4-HMAC-SHA256'
$canonical_querystring += '&X-Amz-Credential=' + [uri]::EscapeDataString(($access_key + '/' + $credential_scope))
$canonical_querystring += '&X-Amz-Date=' + $amz_date
$canonical_querystring += '&X-Amz-Expires=86400'
$canonical_querystring += '&X-Amz-SignedHeaders=' + $signed_headers

$canonical_headers = 'host:' + $host1 + "`n"

$canonical_request = $method + "`n"
$canonical_request += "/" + $key + "`n"
$canonical_request += $canonical_querystring + "`n"
$canonical_request += $canonical_headers + "`n"
$canonical_request += $signed_headers + "`n"
$canonical_request += "UNSIGNED-PAYLOAD"

$algorithm = 'AWS4-HMAC-SHA256'

$canonical_request_hash = hash -request $canonical_request
$string_to_sign = $algorithm + "`n"
$string_to_sign += $amz_date + "`n"
$string_to_sign += $credential_scope + "`n"
$string_to_sign += $canonical_request_hash

$signing_key = getSignatureKey $secret_key $datestamp $region $service
$signature =  HmacSHA256 -secret $signing_key -message $string_to_sign
$signature = ($signature|ForEach-Object ToString x2) -join ''

$canonical_querystring += '&X-Amz-Signature=' + $signature

$request_url = "http://" + $host1 + "/" + $key + "?" + $canonical_querystring

Write-Host $request_url

这篇关于如何在不安装AWS开发工具包的情况下通过Powershell从S3下载文件?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！