将大于40MB的文件上传到Google App Engine？ [英] Upload file bigger than 40MB to Google App Engine?

问题描述

我正在创建一个Google App Engine的网络应用程序来转换10K〜50M的文件。 b 场景：

b
$ b

1. 用户打开 http://fixdeck.appspot.com
   
2. 用户单击浏览，选择文件，提交


3. Servlet将文件加载为InputStream


4. Servlet转换文件
   
5. Servlet将文件保存为OutputStream
   
6. 用户的浏览器接收到转换后的文件并询问在哪里保存，直接作为对步骤2中的请求的响应

（现在我没有实现步骤4，servlet发回文件）

问题：适用于15MB文件，但不适用于40MB文件，说：Error：Request实体太大，你的客户发出的请求太大了。

是否有任何解决方法？

源代码： https://github.com/nicolas-raoul/transdeck

理由： http://code.google.com/p/ankidroid/issues/detail?id=697

解决方案

GAE有一个硬限制。这将限制直接从GAE应用程序上传/下载的大小。

修正的答案（使用Blobstore API。）

Google向 Blobstore API 来处理GAE中较大的文件（最多2GB）。概览文件提供了完整的示例代码。您的网页表单将上传文件到blobstore。然后，blobstore API将POST重写回到您的servlet，您可以在其中进行转换，并将转换后的数据保存到blobstore中（作为新的blob）。

< （不考虑Blobstore作为选项）。

对于下载，我认为GAE唯一的解决方法是将文件打开到服务器上的多个部分，然后在下载后重新组装。虽然这可能无法使用直接的浏览器实现。

（作为替代设计，也许您可​​以将已转换的文件从GAE发送到外部下载位置（例如S3 ）可以在没有GAE限制的情况下通过浏览器下载，我不认为GAE发起的连接有相同的请求/响应大小限制，但是我不积极，不管怎样，你仍然会被限制在最多30秒请求时间。为了解决这个问题，您必须查看 GAE后端实例，并提出了某种异步下载策略。）

为了上传更大的文件，我已经读了使用HTML5 File API的可能性将文件分割成多个块进行上传，然后在服务器上进行重构。示例： http://www.html5rocks.com/en/ tutorials / file / dndfiles /＃toc-slicing-files 。但是，我并不是真正由于改变规格和浏览器功能而导致的解决方案。

I am creating a Google App Engine web app to "transform" files of 10K~50M

Scenario:

1. User opens http://fixdeck.appspot.com in web browser
2. User clicks on "Browse", select file, submits
3. Servlet loads file as an InputStream
4. Servlet transforms file
5. Servlet saves file as an OutputStream
6. The user's browser receives the transformed file and asks where to save it, directly as a response to the request in step 2

(For now I did not implement step 4, the servlet sends the file back without transforming it.)

Problem: It works for 15MB files but not for a 40MB file, saying: "Error: Request Entity Too Large. Your client issued a request that was too large."

Is there any workaround against this?

Source code: https://github.com/nicolas-raoul/transdeck
Rationale: http://code.google.com/p/ankidroid/issues/detail?id=697

解决方案

GAE has a hard limits of 32MB for HTTP requests and HTTP responses. That will limit the size of uploads/downloads directly to/from a GAE app.

Revised Answer (Using Blobstore API.)

Google provides to the Blobstore API for handling larger files in GAE (up to 2GB). The overview documentation provides complete sample code. Your web form will upload the file to blobstore. The blobstore API then rewrites the POST back to your servlet where you can do your transformation and save the transformed data back in to the blobstore (as a new blob).

Original Answer (Didn't Consider Blobstore as an option.)

For downloading, I think GAE only workaround would be to break the file up in to multiple parts on the server, and then reassemble after downloading. That's probably not doable using a straight browser implementation though.

(As an alternative design, perhaps you could send the transformed file from GAE to an external download location (such as S3) where it could be downloaded by the browser without the GAE limit restrictions. I don't believe GAE initiated connections have same request/response size limitations, but I'm not positive. Regardless, you would still be restricted by the 30 second maximum request time. To get around that, you'd have to look in to GAE Backend instances and come up with some sort of asynchronous download strategy.)

For uploading larger files, I've read about the possibility of using HTML5 File APIs to slice the file in to multiple chunks for uploading, and then reconstructing on the server. Example: http://www.html5rocks.com/en/tutorials/file/dndfiles/#toc-slicing-files . However, I don't how practical a solution that really is due to changing specifications and browser capabilities.

这篇关于将大于40MB的文件上传到Google App Engine？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！