使用 HTTP POST 上传 AWS S3 浏览器提供无效签名 [英] AWS S3 browser upload using HTTP POST gives invalid signature

问题描述

我正在开发一个网站，用户应该能够将视频文件上传到 AWS.为了避免不必要的流量，我希望用户直接上传到 AWS(而不是通过 API 服务器).为了不在 JavaScript 中公开我的密钥，我试图在 API 中生成一个签名.但是，当我尝试上传时，它确实告诉我签名不匹配.

I'm working on a website where the users should be able to upload video files to AWS. In order to avoid unnecessary traffic I would like the user to upload directly to AWS (and not through the API server). In order to not expose my secret key in the JavaScript I'm trying to generate a signature in the API. It does, however, tell me when I try to upload, that the signature does not match.

对于签名生成，我一直在使用 http://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-UsingHTTPPOST.html

For signature generation I have been using http://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-UsingHTTPPOST.html

在后端，我运行的是 C#.

On the backend I'm running C#.

我使用

string policy = $@"{{""expiration"":""{expiration}"",""conditions"":[{{""bucket"":""dennisjakobsentestbucket""}},[""starts-with"",""$key"",""""],{{""acl"":""private""}},[""starts-with"",""$Content-Type"",""""],{{""x-amz-algorithm"":""AWS4-HMAC-SHA256""}}]}}";

生成以下内容

{"expiration":"2016-11-27T13:59:32Z","conditions":[{"bucket":"dennisjakobsentestbucket"},["starts-with","$key",""],{"acl":"private"},["starts-with","$Content-Type",""],{"x-amz-algorithm":"AWS4-HMAC-SHA256"}]}

基于 http://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-HTTPPOSTConstructPolicy.html(我对策略进行 base64 编码).我尽量保持非常简单，只是作为一个起点.

based on http://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-HTTPPOSTConstructPolicy.html (I base64 encode the policy). I have tried to keep it very simple, just as a starting point.

为了生成签名，我使用了在 AWS 网站上找到的代码.

For generating the signature, I use code found on the AWS site.

static byte[] HmacSHA256(String data, byte[] key)
{
    String algorithm = "HmacSHA256";
    KeyedHashAlgorithm kha = KeyedHashAlgorithm.Create(algorithm);
    kha.Key = key;

    return kha.ComputeHash(Encoding.UTF8.GetBytes(data));
}

static byte[] GetSignatureKey(String key, String dateStamp, String regionName, String serviceName)
{
    byte[] kSecret = Encoding.UTF8.GetBytes(("AWS4" + key).ToCharArray());
    byte[] kDate = HmacSHA256(dateStamp, kSecret);
    byte[] kRegion = HmacSHA256(regionName, kDate);
    byte[] kService = HmacSHA256(serviceName, kRegion);
    byte[] kSigning = HmacSHA256("aws4_request", kService);

    return kSigning;
}

我是这样使用的:

byte[] signingKey = GetSignatureKey(appSettings["aws:SecretKey"], dateString, appSettings["aws:Region"], "s3");
byte[] signature = HmacSHA256(encodedPolicy, signingKey);

dateString 的格式为 yyyymmdd

where dateString is on the format yyyymmdd

我使用 JavaScript 发布信息

I POST information from JavaScript using

let xmlHttpRequest = new XMLHttpRequest();
let formData = new FormData();
formData.append("key", "<path-to-upload-location>");
formData.append("acl", signature.acl); // private
formData.append("Content-Type", "$Content-Type");
formData.append("AWSAccessKeyId", signature.accessKey);
formData.append("policy", signature.policy); //base64 of policy
formData.append("x-amz-credential", signature.credentials); // <accesskey>/20161126/eu-west-1/s3/aws4_request
formData.append("x-amz-date", signature.date);
formData.append("x-amz-algorithm", "AWS4-HMAC-SHA256");
formData.append("Signature", signature.signature);
formData.append("file", file);

xmlHttpRequest.open("post", "http://<bucketname>.s3-eu-west-1.amazonaws.com/");
xmlHttpRequest.send(formData);

我一直按照 AWS 的规定在任何地方使用 UTF8.在他们的示例中，签名采用十六进制格式，我也尝试过.无论我尝试什么，我都会收到错误 403

I have been using UTF8 everywhere as prescribed by AWS. In their examples the signature is on a hex format, which I have tried as well. No matter what I try I get an error 403

The request signature we calculated does not match the signature you provided. Check your key and signing method.

我在 AWS 上的策略有s3:Get*"、s3:Put*"

My policy on AWS has "s3:Get*", "s3:Put*"

我是否遗漏了某些东西，或者它的工作方式与我预期的完全不同?

Am I missing something or does it just work completely different than what I expect?

下面的答案是步骤之一.另一个是AWS区分大小写十六进制字符串.0xFF != 0xff 在 AWS 眼中.他们想要全小写的签名.

The answer below is one of the steps. The other is that AWS distinguish between upper and lowercase hex strings. 0xFF != 0xff in the eyes of AWS. They want the signature in all lowercase.

推荐答案

您正在使用 Signature Version 4 生成签名，但您正在构建表单，就像您使用 Signature Version 2 一样......好吧，有点.

You are generating the signature using Signature Version 4, but you are constructing the form as though you were using Signature Version 2... well, sort of.

formData.append("AWSAccessKeyId", signature.accessKey);

那是 V2.它根本不应该在这里.

That's V2. It shouldn't be here at all.

formData.append("x-amz-credential", signature.credentials); // <accesskey>/20161126/eu-west-1/s3/aws4_request

这是 V4.请注意这里和上面的 AWS 访问密钥 ID 的冗余提交.这可能是正确的，尽管示例中的大写字母类似于 X-Amz-Credential.

This is V4. Note the redundant submission of the AWS Access Key ID here and above. This one is probably correct, although the examples have capitalization like X-Amz-Credential.

formData.append("x-amz-algorithm", "AWS4-HMAC-SHA256");

这也是正确的，除了它可能需要是X-Amz-Algorithm.(该示例似乎暗示忽略大写).

That is also correct, except it may need to be X-Amz-Algorithm. (The example seems to imply that capitalization is ignored).

formData.append("Signature", signature.signature);

这个是错误的.这应该是 X-Amz-Signature.V4 签名是十六进制的，所以这就是您在这里应该拥有的.V2 签名是 base64.

This one is incorrect. This should be X-Amz-Signature. V4 signatures are hex, so that is what you should have here. V2 signatures are base64.

有一个完整的 V4 示例 此处，它甚至为您提供了一个示例 aws 密钥和机密、日期、区域、存储桶名称等，您可以将其与您的代码一起使用以验证您确实得到了相同的响应.表单实际上不会工作，但重要的问题是您的代码是否可以生成相同的表单、策略和签名.

There's a full V4 example here, which even provides you with an example aws key and secret, date, region, bucket name, etc., that you can use with your code to verify that you indeed get the same response. The form won't actually work but the important question is whether your code can generate the same form, policy, and signature.

对于任何给定的请求，只有一个正确的签名；然而，对于任何给定的策略，可能有不止一种有效的 JSON 编码(由于 JSON 具有空格的灵活性)——但对于任何给定的 JSON 编码，策略只有一种可能的有效 base64 编码.这意味着，如果您的代码使用示例数据生成与示例中显示的表单和签名完全相同的表单和签名，那么它就被证明可以正常工作——这意味着如果您的代码生成相同的表单和策略，则证明它是无效的不同的签名——但还有第三种可能性:如果您的代码生成策略的不同 base64 编码，则测试实际上无法证明您的代码是决定性的，因为这必然会将签名更改为不匹配，但可能仍然是有效的政策.

For any given request, there is only ever exactly one correct signature; however, for any given policy, there may be more than one valid JSON encoding (due to JSON's flexibility with whitespace) -- but for any given JSON encoding there is only one possible valid base64-encoding of the policy. This means that your code, using the example data, is certified as working correctly if it generates exactly the same form and signature as shown in the example -- and it means that your code is proven invalid if it generates the same form and policy with a different signature -- but there is a third possibility: the test actually proves nothing conclusive about your code if your code generates a different base64 encoding of the policy, because that will necessarily change the signature to not match, yet might still be a valid policy.

请注意，Signature V2 仅支持较旧的 S3 区域，而 所有 S3 区域都支持 Signature V4，因此，即使您可以通过让整个签名过程使用 V2 来交替解决此问题，不推荐这样做.

Note that Signature V2 is only suported on older S3 regions, while Signature V4 is supported by all S3 regions, so, even though you could alternately fix this by making your entire signing process use V2, that wouldn't be recommended.

还要注意我们计算的请求签名与您提供的签名不匹配.检查您的密钥和签名方法 不会告诉您有关存储桶 策略或任何用户策略是允许还是拒绝请求的任何信息.此错误不是权限错误.它将在权限检查之前抛出，仅基于签名的有效性，而不是 AWS 访问密钥 ID 是否有权执行请求的操作，这是仅在签名验证后进行测试的内容.

Note also that The request signature we calculated does not match the signature you provided. Check your key and signing method does not tell you anything about whether the bucket policy or any users policies allow or deny the request. This error is not a permissions error. It will be thrown prior to the permissions checks, based solely on the validity of the signature, not whether the AWS Access Key id is authorized to perform the requested operation, which is something that is only tested after the signature is validated.

这篇关于使用 HTTP POST 上传 AWS S3 浏览器提供无效签名的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！