使用Python / Django上传大文件 [英] Uploading large files with Python/Django

问题描述

我想知道，通过使用Django / Python的Web应用程序，上传大小为4GB的文件是否有任何后果？我记得过去使用Java的流媒体上传是首选方法，但是今天还是这样做还是使用Django / Python完全可以这样做？

解决方案

Django将默认将上传的文件数据放入内存中，如果小于2.5MB。任何较大的将被写入服务器的 / tmp 目录，然后在传输完成时复制。许多Django的文件上传设置可以自定义，详细信息可在文档。您还可以自定义文件处理，您一定要这样做。

在考虑任何技术限制之前，使用浏览器上传这样大的文件会给用户一个很差的经验。没有关于传输过程的反馈（尽管Google Chrome的确显示了上传状态的百分比），也没有办法暂停或恢复传输。

您也是可能在服务器上遇到问题。除了处理流数据之外，每个线程都需要很长的时间，所以系统将生成的文件从 / tmp 复制到它的正确位置。

除非您非常有信心，您可以预见服务器可能遇到上传的任何问题，我建议这是一个坏主意。很难通过谷歌找到任何有关这方面的信息，似乎有很多点击描述大文件上传的问题。

虽然Django在技术上有能力接收上传的文件这么大，用户体验和技术难度很差，这可能不是最好的方法。您是否考虑过使用专用软件来处理文件传输？

I am wondering if there are any ramifications in uploading files that are roughly 4GB in size through a web app using Django/Python? I remember in the past streaming uploads using Java was the preferred method but does this still today or is it perfectly safe to do so with Django/Python?

解决方案

Django will by default, put uploaded file data into memory if it is less than 2.5MB. Anything larger will be written to the server's /tmp directory and then copied across when the transfer completes. Many of Django's file upload settings can be customised, details are available in the documentation. You can also customise the file handling and you'll certainly want to do this.

Before we consider any technical constraints, uploading such large files with the browser will give the user a very poor experience. There is no feedback about how the transfer is going (although google chrome does display the upload status as a percentage) and no way to pause or resume transfers.

You are also likely to run into problems on the server. Apart from the extremely long time that each thread will be taken with dealing with the streamed data, you have the time it takes for the system to copy the resulting file from /tmp to its correct location.

Unless you are very confident that you can foresee any problem that the server might have with the uploads, I would suggest that this is a bad idea. It's pretty hard to find any information on this via google and there do seem to be a lot of hits that describe problems with large file uploads.

While Django is technically capable of receiving uploaded files this large, the very poor user experience and technical difficulties mean this may not be the best approach. Have you considered using dedicated software to handle the file transfer?

这篇关于使用Python / Django上传大文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！