从appengine应用上传文件到谷歌云存储 [英] Upload files to Google cloud storage from appengine app

问题描述

我确信这个问题的答案很简单，但对我来说，这证明是非常令人沮丧的，因为我无法将我找到的任何解决方案放入实用的代码中供我自己使用。

我在app引擎上构建了一个应用程序，让用户上传一个文件，然后由该应用程序执行操作。文件的大小通常在几Mbs左右，在某些情况下可能高达20 Mb左右。这足以触发应用程序引擎中的30秒超时，因此我试图直接按照各种网站（包括此处）中的建议上传到云存储。

上传部分表单如下所示：

postenctype =multipart / form-data>
< input type =filename =coords>
< input type =submitname =submitvalue =Transform>
< / form>

处理程序看起来像这样（我只是从应用程序引擎开始，所以很抱歉这看起来很蠢。

  class TransformPage（BaseHandler）：
 def get（self）：
 self。 render_template（'transform.html'，name = self.request.get（'form'））
 
 def post（self）：
 filename = self.request.POST ['coords' ] .filename 
 filesize = int（self.request.headers ['Content_Length']）
 
 if filesize> 5242880：
 self.redirect（'/ error'）
 else：
 save_to_cloud（file.value，filename）
＃在这里开始处理文件
  

我将文件大小检查作为防止30秒超时的一种非常简单的方法，并且5Mb似乎是我在网络上30秒内可以达到的最大大小。

save_to_cloud（）只是我用Google云存储API编写的一个包装器，看起来像这样：

  def save_to_cloud（f，filename）：
文件名= BUCKET +文件名
 create_file_and_write （f，filename）
  

以上所有的作品，我看到上传的文件以云存储，但正如我上面所说，打破较大的文件。
我已经看到有关使用create_upload_url进行搜索的提示，但是我不太清楚与我的情况有关的情况，特别是将上载文件的内容读入内存，以便我可以将其写入云存储。

如果你知道它是如何完成的，这可能是世界上最简单的事情，但是如果没有人在代码中显示我如何做到这一点，做： - （$ / b
$ b< p> $ p

< p>编辑：@Itamar：这确实是我想要完成的，尽管上传表单中还包含一些其他用户选择的内容，但现在我已将代码更改为

  class TransformPage（BaseHandler）：
 def get（self）：
 upload_url = blobstore.create_upload_url（'/ upload'，gs_bucket_name ='/ my_bucket /'）
 
 
 self.render_template（'transform.html'，
 {'name'：self.request.get（'form'），
 {'upload_url'：upload_url}）
  

但我无法使upload_url显示在模板html，看起来像这样

 < form name =convertaction ={{upload_url}}method = postenctype =multipart / form-data> 
  

解决方案

如果我理解正确， do是从App Engine提供的一种表单，允许用户选择要上传的文件。
由于上传的文件可能很大，因此您不希望在App Engine中处理上传，但会将文件直接上传到Google Cloud Storage。

这可以完成，而且不是太困难。
实际上，这正是 App Engine Python Blobstore文档。
这个例子可能会引起混淆，因为当您想要在Cloud Storage中存储文件时，该示例引用Blobstore - 但没关系 - 从1.7.0版开始，您可以这样做：

  upload_url = blobstore.create_upload_url（'/ upload'，gs_bucket_name ='my_bucket'）
  

（而不是链接示例中的步骤1），并且上传URL将直接上载到云存储。

现在，您的表单操作应该是由 blobstore.create_upload_url 函数返回的字符串 upload_url 。

在表单完成处理（表示文件被上传）后，它将重定向到您传递给该函数的参数（在示例中为 / upload


您不需要编写自己的POST处理函数，就像您在问题中描述的一样。

I'm sure the answer to this question is easy, but for me it's proven to be very frustrating since I can't put any solution I've found into practical code for my own use.

I'm building an app on the app engine that let's the user upload a file, which then gets acted on by the app. The size of the file is typically around a few Mbs, and in some cases upto maybe 20 Mb or so. This is enough to trigger the 30 second timeout in app engine, and I am therefore trying to upload to cloud storage directly as suggested on various websites (including here).

The upload part of my form looks like this:

<form name="convert" action="/transform" method="post" enctype="multipart/form-data">
    <input type="file" name="coords">
    <input type="submit" name="submit" value="Transform">
</form>

And the handler looks like this (I'm only starting out with the app engine, so apologies if this looks stupid.

class TransformPage(BaseHandler):
    def get(self):
    self.render_template('transform.html',name=self.request.get('form'))

    def post(self):     
        filename = self.request.POST['coords'].filename
        filesize = int(self.request.headers['Content_Length'])

        if filesize>5242880:
            self.redirect('/error')
        else:
             save_to_cloud(file.value,filename)
             # Start processing file here

I put in the check for file size as a very simple way of preventing the 30s timeout, and 5Mb seems to be the maximum size I can push through in 30s on my network.

The save_to_cloud() is just a wrapper I wrote around the Google cloud storage API, and looks like this:

def save_to_cloud(f,filename):
    filename = BUCKET + filename                                                          
    create_file_and_write(f,filename)

The above all works, and I see the uploaded files ending up in cloud storage, but as I said above, breaks down for larger files. I've seen hints about using create_upload_url searching around, but I am not good enough to see how that pertains to my case, in particular getting the contents of the uploaded file read in to memory so that I can write it to cloud storage.

This is probably the easiest thing in the world if you know how it's done, but I can't do it without someone showing me in code how it can be done :-(

Thanks

Edit: @Itamar: this is indeed what I am trying to accomplish, although the upload form also contains a few other selections to be made by the user. I have now changed my code to this

class TransformPage(BaseHandler):
    def get(self):
        upload_url =blobstore.create_upload_url('/upload',gs_bucket_name='/my_bucket/')


        self.render_template('transform.html',
                            {'name':self.request.get('form'),                 
                            {'upload_url':upload_url})

But I can't get the upload_url to show up in the template html which looks like this

<form name="convert" action="{{ upload_url }}" method="post" enctype="multipart/form-data">

解决方案

If I understand correctly, what you're trying to do is serve a form from App Engine that allows a user to choose a file to upload. Since the uploaded file may be large, you don't want to handle the upload in App Engine, but have the file uploaded directly to Google Cloud Storage.

This can be done, and it's not too difficult. In fact, this is exactly the example given in the App Engine Python Blobstore docs. It might be confusing that the example refers to Blobstore while you want the file in Cloud Storage - but it's OK - it appears that since version 1.7.0 you can do this:

upload_url = blobstore.create_upload_url('/upload', gs_bucket_name='my_bucket')

(instead of step 1 in the example I linked), and the upload URL will upload straight to Cloud Storage.

Now, your form action should be the string upload_url that was returned by the blobstore.create_upload_url function.

After the form completes processing (meaning the file is uploaded), it will redirect to the argument you passed to that function (in the example- to /upload).

You do not need to write your own POST processing function, as you described in the question.

这篇关于从appengine应用上传文件到谷歌云存储的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！