如何在python中为请求者创建签名的s3 url支付桶 [英] How to create a signed s3 url for requester pays bucket in python

问题描述

我有一个我无法控制的请求者付款方式:

I have a requester pays bucket that I do not control in form:

s3://bucket-name/path-to-my-file

我正在尝试生成一个预签名的URL，以发送到Web应用程序以在浏览器中呈现它.

I am attempting to generate a presigned url to send to a web app to render it in browser.

我已经阅读了 boto s3 documentation ，但找不到任何涵盖此内容的方法:(

I've gone through the boto s3 documentation but can't find anything that covers this :(

以下我的脚本创建了没有访问权限的返回URL，并从s3返回此错误:

My script below creates returns URL that does not have access and returns this error from s3:

<Error>
  <Code>AccessDenied</Code>
  <Message>Access Denied</Message>
  <RequestId>11DCA24D8DF2E9E8</RequestId>
  <HostId>SeTDlt66hPsj5/dV1pOT9GnYyWgpSGI4ezI3wC7iz8Pny9sy2sUWsuUsl4JkEQeBXXIyiE8EXCk=</HostId>
</Error>

我有信心，这是因为存储桶是请求者支付的，因为当我在aws cli中运行此命令时它起作用:

I'm confident this is because the bucket is requester pays, becuase when I run this command in aws cli it works:

aws s3 cp s3://blackfynn-discover-use1/66/2/files/Derivatives . --request-payer requester --recursive

但是这个返回Forbidden:

aws s3 cp s3://blackfynn-discover-use1/66/2/files/Derivatives . 

这是我的python脚本，如果不是请求者付款，它将起作用:

Here's my python script which would work if it was not requester pays:

import requests
import boto3

def get_signed_url(s3_url):
    # Get the service client.
    s3 = boto3.client('s3')
    bucket_name, key_name = split_s3_bucket_key(s3_url)
    # Generate the URL to get 'key-name' from 'bucket-name'
    url = s3.generate_presigned_url(
        ClientMethod='get_object',
        Params={
            'Bucket': bucket_name,
            'Key': key_name
        }
    )
    return url

def split_s3_bucket_key(s3_path):
    """Split s3 path into bucket and key prefix.
    This will also handle the s3:// prefix.
    :return: Tuple of ('bucketname', 'keyname')
    """
    if s3_path.startswith('s3://'):
        s3_path = s3_path[5:]
    return find_bucket_key(s3_path)

def find_bucket_key(s3_path):
    """
    This is a helper function that given an s3 path such that the path is of
    the form: bucket/key
    It will return the bucket and the key represented by the s3 path
    """
    s3_components = s3_path.split('/')
    bucket = s3_components[0]
    s3_key = ""
    if len(s3_components) > 1:
        s3_key = '/'.join(s3_components[1:])
    return bucket, s3_key

s3_file_path = 's3://blackfynn-discover-use1/66/2/files/Derivatives/manifest.xlsx'
get_signed_url(s3_file_path)

推荐答案

URL似乎需要包含x-amz-request-payer=requester，但这在创建预签名URL时也可能需要指定.

It looks like the URL will need to include x-amz-request-payer=requester, but this might also need to be specified when creating the pre-signed URL.

尝试下面显示的建议，然后让我们知道它是否对您有用！

Try the advice shown below, then let us know whether it worked for you!

从在请求者付费存储桶中下载对象-Amazon Simple Storage Service :

对于签名的URL，请在请求中加入x-amz-request-payer=requester

来自支持请求者支付S3存储桶·问题346·samtools/htslib :

好的，能够在具有良好libcurl支持的情况下编译htslib.确认可以使用预签名的URL来查看文件:

OK, was able to compile htslib with good libcurl support. Confirmed that it can take a presigned URL to view files:

import boto3
client = boto3.client('s3')
url  = client.generate_presigned_url("get_object", Params={"Bucket":"angel-reqpay","Key":"test.cram" , "RequestPayer":'requester'})

来自 AWS开发人员论坛:宣布... :

您的网址如下所示:

You URL would look something like:

http://somebucket.s3.amazonaws.com/key/[.....]&x-amz-request-payer=requester

这篇关于如何在python中为请求者创建签名的s3 url支付桶的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！