在 Google Cloud Storage 签名 URL 中提供回调 URL [英] Provide a callback URL in Google Cloud Storage signed URL

问题描述

使用 BlobStore 的 createUploadURL 函数，我可以提供一个回调连同将发布到回调 URL 的标头数据.

When uploading to GCS (Google Cloud Storage) using the BlobStore's createUploadURL function, I can provide a callback together with header data that will be POSTed to the callback URL.

似乎没有办法使用 GCS 的 签名网址的

There doesn't seem to be a way to do that with GCS's signed URL's

我知道有 对象更改通知，但不会允许用户在 POST 的标头中提供上传特定信息，这可以通过 createUploadURL 的回调实现.

I know there is Object Change Notification but that won't allow the user to provide upload specific information in the header of a POST, the way it is possible with createUploadURL's callback.

我的感觉是，如果 createUploadURL 可以做到，那么一定有办法使用签名 URL 来做到这一点，但我找不到任何文档.我想知道是否有人知道 createUploadURL 是如何实现回调调用行为的.

My feeling is, if createUploadURL can do it, there must be a way to do it with signed URL's, but I can't find any documentation on it. I was wondering if anyone may know how createUploadURL achieves that callback calling behavior.

PS:由于它创建的 __BlobInfo__ 实体，我正试图从 createUploadURL 移开，对于我的特定用例我不需要，并且不知何故似乎不可磨灭并且正在浪费存储空间空间.

PS: I'm trying to move away from createUploadURL because of the __BlobInfo__ entities it creates, which for my specific use case I do not need, and somehow seem to be indelible and are wasting storage space.

更新:成功了！方法如下:

Update: It worked! Here is how:

简答:不能用 PUT，但可以使用 POST

Short Answer: It cannot be done with PUT, but can be done with POST

长答案:

如果您查看 签名 URL 页面，在 HTTP_Verb 前面，Description 下，有一个微妙的说明，这个页面只与 GET、HEAD、PUT 和 DELETE 相关，但 POST 是完全不同的游戏.我错过了这一点，但事实证明它非常重要.

If you look at the signed-URL page, in front of HTTP_Verb, under Description, there is a subtle note that this page is only relevant to GET, HEAD, PUT, and DELETE, but POST is a completely different game. I had missed this, but it turned out to be very important.

有一整页 HTTP Headers 没有列出重要的可以与 POST 一起使用的标头；该标头是success_action_redirect，因为voscausa 正确回答.

There is a whole page of HTTP Headers that does not list an important header that can be used with POST; that header is success_action_redirect, as voscausa correctly answered.

在 POST 页面中，Google强烈建议"使用 PUT，除非处理表单数据.但是，POST 有一些 PUT 没有的好功能.他们可能会担心 POST 给了我们太多的字符串来吊死自己.

In the POST page Google "strongly recommends" using PUT, unless dealing with form data. However, POST has a few nice features that PUT does not have. They may worry that POST gives us too many strings to hang ourselves with.

但我会说完全值得放弃 createUploadURL，并编写自己的代码来重定向到回调.方法如下:

But I'd say it is totally worth dropping createUploadURL, and writing your own code to redirect to a callback. Here is how:

代码:

如果您使用 Python voscausa 的 代码很有帮助.

If you are working in Python voscausa's code is very helpful.

我正在使用 apejs 在 Java 应用程序中编写 javascript，所以我的代码如下所示:

I'm using apejs to write javascript in a Java app, so my code looks like this:

            var exp = new Date()
            exp.setTime(exp.getTime() + 1000 * 60 * 100); //100 minutes

            json['GoogleAccessId'] = String(appIdentity.getServiceAccountName())
            json['key'] = keyGenerator()
            json['bucket'] = bucket
            json['Expires'] = exp.toISOString(); 
            json['success_action_redirect'] = "https://" + request.getServerName() + "/test2/";
            json['uri'] = 'https://' + bucket + '.storage.googleapis.com/'; 

            var policy = {'expiration': json.Expires
                        , 'conditions': [
                             ["starts-with", "$key", json.key],
                             {'Expires': json.Expires},
                             {'bucket': json.bucket},
                             {"success_action_redirect": json.success_action_redirect}
                           ]
                        };

            var plain = StringToBytes(JSON.stringify(policy))
            json['policy'] = String(Base64.encodeBase64String(plain))
            var result = appIdentity.signForApp(Base64.encodeBase64(plain, false));
            json['signature'] = String(Base64.encodeBase64String(result.getSignature()))

上面的代码首先提供了相关的字段.然后创建一个策略对象.然后它将对象字符串化并将其转换为字节数组(您可以在 Java 中使用 .getBytes.我必须为 javascript 编写一个函数).此数组的 ba​​se64 编码版本填充​​ policy 字段.然后使用 appidentity 包对其进行签名.最后将签名进行base64编码，我们就完成了.

The code above first provides the relevant fields. Then creates a policy object. Then it stringify's the object and converts it into a byte array (you can use .getBytes in Java. I had to write a function for javascript). A base64 encoded version of this array, populates the policy field. Then it is signed using the appidentity package. Finally the signature is base64 encoded, and we are done.

在客户端，json 对象的所有成员都将添加到表单中，除了 uri 是表单的地址.

On the client side, all members of the json object will be added to the Form, except the uri which is the form's address.

        var formData = new FormData(document.forms.namedItem('upload'));
        var blob = new Blob([thedata], {type: 'application/json'})
        var keys = ['GoogleAccessId', 'key', 'bucket', 'Expires', 'success_action_redirect', 'policy', 'signature']
        for(field in keys)
          formData.append(keys[field], url[keys[field]])
        formData.append('file', blob)
        var rest = new XMLHttpRequest();
        rest.open('POST', url.uri)
        rest.onload = callback_function
        rest.send(formData)

如果您不提供重定向，则响应状态将为 204 表示成功.但是，如果您进行重定向，则状态将为 200.如果您收到 403 或 400，则有关签名或策略的某些内容可能是错误的.查看响应文本.如果经常有帮助.

If you do not provide a redirect, the response status will be 204 for success. But if you do redirect, the status will be 200. If you got 403 or 400 something about the signature or policy maybe wrong. Look at the responseText. If is often helpful.

需要注意的几点:

• POST 和 PUT 都有一个签名字段，但它们的含义略有不同.在 POST 的情况下，这是 policy 的签名.
• PUT 有一个包含键(对象名称)的 baseurl，但用于 POST 的 URL 可能只包含存储桶名称
• PUT 要求从 UNIX 纪元开始以秒为单位过期，但 POST 需要它作为 ISO 字符串.
• PUT 签名应该是 URL 编码的(Java:通过调用 URLEncoder.encode 来包装它).但是对于 POST，Base64 编码就足够了.
• 通过扩展，对于 POST 执行 Base64.encodeBase64String(result.getSignature())，并且不要使用 Base64.encodeBase64URLSafeString 函数
• 您不能通过 POST 传递额外的标头；仅允许在 POST 页面中列出的内容.
• 如果您为 success_action_redirect 提供 URL，它将收到带有 key、bucket 和 eTag 的 GET.
• 使用 POST 的另一个好处是可以提供大小限制.但是，使用 PUT，如果文件违反了您的大小限制，您只能在完全上传后将其删除，即使它是多 TB.

• Both POST and PUT have a signature field, but these mean slightly different things. In case of POST, this is a signature of the policy.
• PUT has a baseurl which contains the key (object name), but the URL used for POST may only include bucket name
• PUT requires expiration as seconds from UNIX epoch, but POST wants it as an ISO string.
• A PUT signature should be URL encoded (Java: by wrapping it with a URLEncoder.encode call). But for POST, Base64 encoding suffices.
• By extension, for POST do Base64.encodeBase64String(result.getSignature()), and do not use the Base64.encodeBase64URLSafeString function
• You cannot pass extra headers with the POST; only those listed in the POST page are allowed.
• If you provide a URL for success_action_redirect, it will receive a GET with the key, bucket and eTag.
• The other benefit of using POST is you can provide size limits. With PUT however, if a file breached your size restriction, you can only delete it after it was fully uploaded, even if it is multiple-tera-bytes.

createUploadURL 有什么问题?

上面的方法是手动的createUploadURL.但是:

The method above is a manual createUploadURL. But:

• 您不会得到那些创建许多索引并且不可磨灭的 __BlobInfo__ 对象.这让我很恼火，因为它浪费了大量空间(这让我想起了一个单独的问题:issue 4231.请给它一颗星)
• 您可以提供自己的对象名称，这有助于在您的存储桶中创建文件夹.
• 您可以为每个链接提供不同的到期日期.

• You don't get those __BlobInfo__ objects which create many indexes and are indelible. This irritates me as it wastes a lot of space (which reminds me of a separate issue: issue 4231. Please go give it a star)
• You can provide your own object name, which helps create folders in your bucket.
• You can provide different expiration dates for each link.

对于极少数的 javascript 应用工程师:

For the very very few javascript app-engineers:

function StringToBytes(sz) {
  map = function(x) {return x.charCodeAt(0)}
  return sz.split('').map(map)
}

推荐答案

您可以在使用 GCS post 对象时将 succes_action_redirect 包含在策略文档中.

You can include succes_action_redirect in a policy document when you use GCS post object.

此处的文档:文档:https://cloud.google.com/storage/docs/xml-api/post-object
这里的 Python 示例:https://github.com/voscausa/appengine-gcs-upload

回调结果示例:

def ok(self):
    """ GCS upload success callback """

    logging.debug('GCS upload result : %s' % self.request.query_string)
    bucket = self.request.get('bucket', default_value='')
    key = self.request.get('key', default_value='')
    key_parts = key.rsplit('/', 1)
    folder = key_parts[0] if len(key_parts) > 1 else None

这篇关于在 Google Cloud Storage 签名 URL 中提供回调 URL的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！