使用 AWS S3 REST API 在不知道存储桶区域的情况下检索存储桶的对象 [英] Retrieve bucket's objects without knowing bucket's region with AWS S3 REST API

问题描述

我正在尝试使用 AWS S3 REST API 检索存储桶的对象(我没有使用开发工具包)，但不幸的是我面临以下问题:当我调用API.

I'm trying to retrieve a bucket's objects with AWS S3 REST API (I'm not using the SDK) but unfortunately I'm facing the following problem : I don't know its region when I do the call to the API.

这就是我所做的:

• 我向https://[bucketname] .s3.amazonaws.com/"发出请求
• 我使用设置为us-west-1"的默认区域签署请求(使用 AWS4 签名流程)
• 但是假设我尝试 GET 的存储桶设置为区域us-west-2"，AWS 向我抛出错误

因此这是我的问题:

有没有办法在不知道存储桶区域的情况下获取存储桶的对象列表?或者有什么简单的方法可以获取桶的区域?

Is there any way to get a list of the objects of a bucket without knowing its region ? Or is there any simple way to get the region of a bucket ?

信息:我读过有两种样式可以调用存储桶，路径样式"和虚拟托管样式"，但是每当我将请求发送到https://s3.amazonaws.com/ [bucketname]" 相反，它给了我一个重定向永久错误...

INFO : I've read there are two style to call a bucket, "path-style" and "virtual-hosted-style", but whenever I send the request to "https://s3.amazonaws.com/ [bucketname]" instead it gives me a redirectpermanent error...

推荐答案

AWS V4 身份验证要求您知道存储桶的区域，以便您可以正确签署请求.否则:

AWS V4 authentication requires that you know the bucket's region so that you can correctly sign the request. Otherwise:

HTTP/1.1 400 Bad Request

<?xml version="1.0" encoding="UTF-8"?>
<Error>
  <Code>AuthorizationHeaderMalformed</Code>
  <Message>The authorization header is malformed; the region 'us-east-1' is wrong; expecting 'us-west-2'</Message>
  <Region>us-west-2</Region>
  <RequestId>xxxx</RequestId>
  <HostId>xxxx</HostId>
</Error>

V2 签名不是特定于区域的，因此以前有一种简单的方法可以使用 V2 请求了解存储桶的区域:

V2 signatures were not region-specific, so there was previously a simple way you could learn the bucket's region using a V2 request:

http://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketGETlocation.html.html

但是，V4 似乎有一个 catch-22，因为您必须先知道存储桶的区域，然后才能进行调用以发现存储桶的区域.

However, there apprears to be a catch-22 with V4, since you have to know the bucket's region before you can make the call to discover the bucket's region.

然后，第一个解决方案是捕获错误响应中返回的 并使用该区域为存储桶的未来请求签名.显然，您希望缓存此信息，因为不这样做会降低性能并增加成本.

The first solution, then, is to capture the <Region> returned in the error response and use that region for signing future requests for the bucket. Obviously, you'd want to cache this information, since not doing so would lower performance and increase costs.

或者，有一种方法可以仅使用此 URL 格式向美国标准区域 (us-east-1) 询问任何存储区在任何区域中的位置:

Alternately, there is a way to ask the US-Standard region (us-east-1) about the location of any bucket, in any region, using this URL format only:

https://s3.amazonaws.com/bucket-name?location

使用 us-east-1 区域签署此请求，无论存储桶位于何处，您都会收到响应.请注意，如果存储桶在 us-east-1(美国标准)中，LocationConstraint 将返回空.

Sign this request with the us-east-1 region and you will get a response, wherever the bucket happens to be. Note that if the bucket is in us-east-1 (US-Standard) the LocationConstraint is returned empty.

<?xml version="1.0" encoding="UTF-8"?>
<LocationConstraint xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  us-west-2
</LocationConstraint>

您不能使用此 URL 构造来实际列出其他地区的存储桶对象，因为 https://s3.amazonaws.com/ 始终将您的请求路由到美国标准 (us-east-1) 但您可以使用它来发现任何存储区的区域，因为美国标准具有可用的信息.

You cannot use this URL construct for actually listing bucket objects in other regions, since https://s3.amazonaws.com/ always routes your request to US-Standard (us-east-1) but you can use it to discover the region of any bucket, since US-Standard has that information available.

更新/附加

在某些时候，S3 似乎添加了一个新的响应头，x-amx-bucket-region:，它似乎是对 REST API 的一个未记录的添加，并且似乎被添加到许多 S3 错误响应中，尤其是 403 错误.

At some point, S3 appears to have added a new response header, x-amx-bucket-region: which appears to be an undocumented addition to the REST API, and appears to be added to many S3 error responses, particularly 403 errors.

如果您收到错误响应，这似乎是一种有用的自我纠正机制.

This seems like a useful mechanism for self-correction if you receive an error response.

此外，上述有关询问美国标准区域有关任何位置的任何存储桶位置的信息仍然是准确的，如果发送到任何区域 S3 REST 端点(而不仅仅是美国标准)，相同的请求应该可以工作，因为所有区域知道所有其他桶的位置:例如https://s3-us-west-2.amazonaws.com/bucket-name?location 会询问 US-West-2(俄勒冈)，它也有相同的可用信息，用于任何存储桶全球.在某些情况下，询问离您最近的地区可能是可取的.

Also, the information above about interrogating the US-Standard region about the location of any bucket anywhere is still accurate, and the same request should work if sent to any regional S3 REST endpoint, not just US-Standard, since all regions are aware of the location of all other buckets: e.g. https://s3-us-west-2.amazonaws.com/bucket-name?location would ask US-West-2 (Oregon), which also has the same information available, for any bucket globally. It might be desirable in some cases to interrogate your nearest region.

这篇关于使用 AWS S3 REST API 在不知道存储桶区域的情况下检索存储桶的对象的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！