直接上传到S3使用Python /博托/ Django的构建策略 [英] Direct Upload to S3 Using Python/Boto/Django to Construct Policy

问题描述

我仔细观察过这个问题，多次迭代，到目前为止，搜索出许多不同的例子，并已全部通过的文件。

I have been through many iterations of this problem so far, searched out many different examples, and have been all through the documentation.

我想Plupload（http://www.plupload.com/）结合AWS的S3直接交法（http://aws.amazon.com/articles/1434）。不过，我相信有什么毛病，我构建我的发送策略和签名的方式。当我提交表单，我没有得到来自服务器的响应，而是我与服务器的连接被重置。

I am trying to combine Plupload (http://www.plupload.com/) with the AWS S3 direct post method (http://aws.amazon.com/articles/1434). However, I believe there's something wrong with the way I am constructing my policy and signature for transmission. When I submit the form, I don't get a response from the server, but rather my connection to the server is reset.

我已经使用Python code的例子尝试：

I have attempted using the python code in the example:

import base64
import hmac, sha

policy = base64.b64encode(policy_document)

signature = base64.b64encode(
hmac.new(aws_secret_key, policy, sha).digest())

我也曾尝试使用更先进的最新hashlib库的蟒蛇。不管方法我用它来构建我的政策和签名，我总是得到不同的值比这里生成的：

I have also tried to use the more up-to-date hashlib library in python. Whatever method I use to construct my policy and signature, I always get different values than those generated here:

http://s3.amazonaws.com /doc/s3-example-$c$c/post/post_sample.html

我已经经历了这个问题阅读：

I have read through this question:

<一个href="http://stackoverflow.com/questions/4639471/how-do-i-make-plupload-upload-directly-to-amazon-s3">How让我的Plupload直接上传到亚马逊的S3？

但我发现提供过于复杂的例子，并没有能够准确地贯彻落实。

But I found the examples provided to be overly complicated and wasn't able to accurately implement them.

我最近的尝试已经使用博托库的部分：

My most recent attempts have been to use portions of the boto library:

http://boto.cloudhackers.com/ref/ s3.html＃模块boto.s3.connection

但使用的方法一直没有为我工作的S3Commection.build_post_form_args无论是。

But using the S3Commection.build_post_form_args method has not worked for me either.

如果任何人都可以提供如何创建使用python的形式后，我非常AP preciate它一个恰当的例子。为什么连接总是复位即使是一些简单的见解将是很好。

If anyone could provide a proper example of how to create the post form using python, I would very much appreciate it. Even some simple insights on why the connection is always reset would be nice.

一些注意事项：

我想用hashlib如果可能的话。 我想从亚马逊的XML响应（presumablysuccess_action_status ='201'做到这一点） 我需要能够上传稍大类型的文件，最大尺寸〜2GB。

I would like to use hashlib if possible. I want to get an XML response from Amazon (presumably "success_action_status = '201'" does this) I need to be able to upload largish type files, max size ~2GB.

最后一个音符，当我运行这个在Chrome浏览器，它提供了上传进度，并上传失败，通常约37％。

One final note, when I run this in Chrome, it provides upload progress, and the upload usually fails around 37%.

推荐答案

弥敦道的回答帮助让我开始。我已经包括了两种解决方案当前已为我工作。

Nathan's answer helped get me started. I've included two solutions that are currently working for me.

第一个解决方案是使用普通的Python。第二个使用博托。

The first solution uses plain Python. The second uses boto.

我试图让博托工作第一，但一直得到错误。于是我又回到了亚马逊Ruby文档并获得S3接受使用python没有博托文件。 （浏览器上传至S3使用HTML POST ）

I tried to get boto working first, but kept getting errors. So I went back to the Amazon ruby documentation and got S3 to accept files using python without boto. (Browser Uploads to S3 using HTML POST)

了解了事情的原委后，我能解决我的错误，并使用宝途，这是一个简单的解决方案。

After understanding what was going on, I was able to fix my errors and use boto, which is a simpler solution.

我在其中的解决方案1，因为它明确地表明了政策文件和签名如何设置使用python。

I'm including solution 1 because it shows explicitly how to setup the policy document and signature using python.

我的目标是创建HTML上传页面是一个动态的页面，伴随着成功页面，用户可以看到一个上传成功后。解决方案1显示的形式上传页面的动态创建，而解决方案2显示了创建两个上传表单页面，并成功页面。

My goal was to create the html upload page as a dynamic page, along with the "success" page the user sees after a successful upload. Solution 1 shows the dynamic creation of the form upload page, while solution 2 shows the creation of both the upload form page and the success page.

解决方案1：

import base64
import hmac, hashlib

###### EDIT ONLY THE FOLLOWING ITEMS ######

DEBUG = 1
AWS_SECRET_KEY = "MySecretKey"
AWS_ACCESS_KEY = "MyAccessKey"
HTML_NAME = "S3PostForm.html"
EXPIRE_DATE = "2015-01-01T00:00:00Z" # Jan 1, 2015 gmt
FILE_TO_UPLOAD = "${filename}"
BUCKET = "media.mysite.com"
KEY = ""
ACL = "public-read" # or "private"
SUCCESS = "http://media.mysite.com/success.html"
CONTENT_TYPE = ""
CONTENT_LENGTH = 1024**3 # One gigabyte
HTTP_OR_HTTPS = "http" # Or "https" for better security
PAGE_TITLE = "My Html Upload to S3 Form"
ACTION = "%s://%s.s3.amazonaws.com/" % (HTTP_OR_HTTPS, BUCKET)

###### DON'T EDIT FROM HERE ON DOWN ######

policy_document_data = {
"expire": EXPIRE_DATE,
"bucket_name": BUCKET,
"key_name": KEY,
"acl_name": ACL,
"success_redirect": SUCCESS,
"content_name": CONTENT_TYPE,
"content_length": CONTENT_LENGTH,
}

policy_document = """
{"expiration": "%(expire)s",
  "conditions": [ 
    {"bucket": "%(bucket_name)s"}, 
    ["starts-with", "$key", "%(key_name)s"],
    {"acl": "%(acl_name)s"},
    {"success_action_redirect": "%(success_redirect)s"},
    ["starts-with", "$Content-Type", "%(content_name)s"],
    ["content-length-range", 0, %(content_length)d]
  ]
}
""" % policy_document_data

policy = base64.b64encode(policy_document)
signature = base64.b64encode(hmac.new(AWS_SECRET_KEY, policy, hashlib.sha1).digest())

html_page_data = {
"page_title": PAGE_TITLE,
"action_name": ACTION,
"filename": FILE_TO_UPLOAD,
"access_name": AWS_ACCESS_KEY,
"acl_name": ACL,
"redirect_name": SUCCESS,
"policy_name": policy,
"sig_name": signature,
"content_name": CONTENT_TYPE,
}

html_page = """
<html> 
 <head>
  <title>%(page_title)s</title> 
  <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
 </head>
<body>
 <form action="%(action_name)s" method="post" enctype="multipart/form-data">
  <input type="hidden" name="key" value="%(filename)s">
  <input type="hidden" name="AWSAccessKeyId" value="%(access_name)s">
  <input type="hidden" name="acl" value="%(acl_name)s">
  <input type="hidden" name="success_action_redirect" value="%(redirect_name)s">
  <input type="hidden" name="policy" value="%(policy_name)s">
  <input type="hidden" name="signature" value="%(sig_name)s">
  <input type="hidden" name="Content-Type" value="%(content_name)s">

  <!-- Include any additional input fields here -->

  Browse to locate the file to upload:<br \> <br \>

  <input name="file" type="file"><br> <br \>
  <input type="submit" value="Upload File to S3"> 
 </form> 
</body>
</html>
""" % html_page_data

with open(HTML_NAME, "wb") as f:
    f.write(html_page)

###### Dump output if testing ######
if DEBUG:

    if 1: # Set true if not using the LEO editor
        class G:
            def es(self, data):print(data)
        g = G()

    items = [
    "",
    "",
    "policy_document: %s" % policy_document,
    "ploicy: %s" % policy,
    "signature: %s" % signature,
    "",
    "",
    ]
    for item in items:
        g.es(item)

解决方案2：

Solution 2:

from boto.s3 import connection

###### EDIT ONLY THE FOLLOWING ITEMS ######

DEBUG = 1
AWS_SECRET_KEY = "MySecretKey"
AWS_ACCESS_KEY = "MyAccessKey"
HTML_NAME = "S3PostForm.html"
SUCCESS_NAME = "success.html"
EXPIRES = 60*60*24*356 # seconds = 1 year
BUCKET = "media.mysite.com"
KEY = "${filename}" # will match file entered by user
ACL = "public-read" # or "private"
SUCCESS = "http://media.mysite.com/success.html"
CONTENT_TYPE = "" # seems to work this way
CONTENT_LENGTH = 1024**3 # One gigabyte
HTTP_OR_HTTPS = "http" # Or https for better security
PAGE_TITLE = "My Html Upload to S3 Form"

###### DON'T EDIT FROM HERE ON DOWN ######

conn = connection.S3Connection(AWS_ACCESS_KEY,AWS_SECRET_KEY)
args = conn.build_post_form_args(
    BUCKET,
    KEY,
    expires_in=EXPIRES,
    acl=ACL,
    success_action_redirect=SUCCESS,
    max_content_length=CONTENT_LENGTH,
    http_method=HTTP_OR_HTTPS,
    fields=None,
    conditions=None,
    storage_class='STANDARD',
    server_side_encryption=None,
    )

form_fields = ""
line = '  <input type="hidden" name="%s" value="%s" >\n'
for item in args['fields']:
    new_line = line % (item["name"], item["value"])
    form_fields += new_line

html_page_data = {
"page_title": PAGE_TITLE,
"action": args["action"],
"input_fields": form_fields,
}

html_page = """
<html> 
 <head>
  <title>%(page_title)s</title> 
  <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
 </head>
<body>
 <form action="%(action)s" method="post" enctype="multipart/form-data" >
%(input_fields)s
  <!-- Include any additional input fields here -->

  Browse to locate the file to upload:<br \> <br \>

  <input name="file" type="file"><br> <br \>
  <input type="submit" value="Upload File to S3"> 
 </form> 
</body>
</html>
""" % html_page_data

with open(HTML_NAME, "wb") as f:
    f.write(html_page)

success_page = """
<html>
  <head>
    <title>S3 POST Success Page</title>
      <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
      <script src="jquery.js"></script>
      <script src="purl.js"></script>
<!--

    Amazon S3 passes three data items in the url of this page if
        the upload was successful:
        bucket = bucket name
        key = file name upload to the bucket
        etag = hash of file

    The following script parses these values and puts them in
    the page to be displayed.

-->

<script type="text/javascript">
var pname,url,val,params=["bucket","key","etag"];
$(document).ready(function()
{
  url = $.url();
  for (param in params)
  {
    pname = params[param];
    val = url.param(pname);
    if(typeof val != 'undefined')
      document.getElementById(pname).value = val;
  }
});
</script>

  </head>
  <body>
      <div style="margin:0 auto;text-align:center;">
      <p>Congratulations!</p>
      <p>You have successfully uploaded the file.</p>
        <form action="#" method="get"
          >Location:
        <br />
          <input type="text" name="bucket" id="bucket" />
        <br />File Name:
        <br />
          <input type="text" name="key" id="key" />
        <br />Hash:
        <br />
          <input type="text" name="etag" id="etag" />
      </form>
    </div>
  </body>
</html>
"""

with open(SUCCESS_NAME, "wb") as f:
    f.write(success_page)

###### Dump output if testing ######
if DEBUG:

    if 1: # Set true if not using the LEO editor
        class G:
            def es(self, data):print(data)
        g = G()

    g.es("conn = %s" % conn)
    for key in args.keys():
        if key is not "fields":
            g.es("%s: %s" % (key, args[key]))
            continue
        for item in args['fields']:
            g.es(item)

这篇关于直接上传到S3使用Python /博托/ Django的构建策略的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！