提供大文件的Django Filewrapper内存错误，如何流式传输 [英] Django Filewrapper memory error serving big files, how to stream

问题描述

我有这样的代码：

@login_required
def download_file(request):
    content_type = "application/octet-stream"
    download_name = os.path.join(DATA_ROOT, "video.avi")

    with open(download_name, "rb") as f:
        wrapper = FileWrapper(f, 8192)
        response = HttpResponse(wrapper, content_type=content_type)
    response['Content-Disposition'] = 'attachment; filename=blabla.avi'
    response['Content-Length'] = os.path.getsize(download_name)
    # response['Content-Length'] = _file.size
    return response

似乎可行。但是，如果我下载更大的文件（例如〜600MB），则内存消耗将增加600MB。经过几次这样的下载后，我的服务器抛出了一个错误：

It seems that it works. However, If I download bigger file (~600MB for example) my memory consumption increase by this 600MB. After few such a downloads my server throws:

内部服务器错误：/ download / Traceback（最近一次调用）：

文件
/home/matous/.local/lib/python3.5/site-packages/django/core/handlers/exception.py\"，
第35行，位于内部
response = get_response（request）文件 /home/matous/.local/lib/python3.5/site-packages/django/core/handlers/base.py，
第128行，在_get_response $ b $中b响应= self.process_exception_by_middleware（e，request）文件 /home/matous/.local/lib/python3.5/site-packages/django/core/handlers/base.py\"，
行126，在_get_response
response = wraped_callback（request，* callback_args，** callback_kwargs）文件 /home/matous/.local/lib/python3.5/site-packages/django/contrib/auth/decorators.py，
第21行，在_wrapped_view
中，返回view_func（request，* args，** kwargs）File / media / matous / 89104d3d-fa52-4b14-9c5d-9ec54ceebebb / home / matous / phd / emoapp / emoapp / mainapp /views.py\"，
行118，在download_file
response = HttpResponse（wrapper，content_type = content_type）文件 /home/matous/.local/lib/python3.5/site-packages/django /http/response.py，
行285，在 init 中，
self.content = content文件 /home/matous/.local/lib/python3.5/site -packages / django / http / response.py，
行308，内容为
content = b''。join（self.make_bytes（chunk）表示值的块）MemoryError

Internal Server Error: /download/ Traceback (most recent call last):
File "/home/matous/.local/lib/python3.5/site-packages/django/core/handlers/exception.py", line 35, in inner response = get_response(request) File "/home/matous/.local/lib/python3.5/site-packages/django/core/handlers/base.py", line 128, in _get_response response = self.process_exception_by_middleware(e, request) File "/home/matous/.local/lib/python3.5/site-packages/django/core/handlers/base.py", line 126, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/home/matous/.local/lib/python3.5/site-packages/django/contrib/auth/decorators.py", line 21, in _wrapped_view return view_func(request, *args, **kwargs) File "/media/matous/89104d3d-fa52-4b14-9c5d-9ec54ceebebb/home/matous/phd/emoapp/emoapp/mainapp/views.py", line 118, in download_file response = HttpResponse(wrapper, content_type=content_type) File "/home/matous/.local/lib/python3.5/site-packages/django/http/response.py", line 285, in init self.content = content File "/home/matous/.local/lib/python3.5/site-packages/django/http/response.py", line 308, in content content = b''.join(self.make_bytes(chunk) for chunk in value) MemoryError

我在做什么错？

注意：我知道不应该提供大文件，是否可以通过某种方式对其进行配置，使其从硬盘驱动器逐流传输？

Note: I know that big files should not be served by Django, but I am looking for simple approach that allows to verify user access rights for any served file.

推荐答案

尝试使用 StreamingHttpResponse 相反，这会有所帮助，这正是您要寻找的东西。

Try to use StreamingHttpResponse instead, that will help, it is exactly what you are looking for.

是否可能配置它以某种方式将其从硬盘驱动器逐个流传输而没有这种疯狂的内存存储？

import os
from django.http import StreamingHttpResponse
from django.core.servers.basehttp import FileWrapper #django <=1.8
from wsgiref.util import FileWrapper #django >1.8

@login_required
def download_file(request):
   file_path = os.path.join(DATA_ROOT, "video.avi")
   filename = os.path.basename(file_path)
   chunk_size = 8192
   response = StreamingHttpResponse(
       FileWrapper(open(file_path, 'rb'), chunk_size),
       content_type="application/octet-stream"
   )
   response['Content-Length'] = os.path.getsize(file_path)    
   response['Content-Disposition'] = "attachment; filename=%s" % filename
   return response

这将分批传输文件，而不会将其加载到内存中；或者，您可以使用 FileResponse

This will stream your file in chunks without loading it in memory; alternatively, you can use FileResponse,

这是 StreamingHttpResponse 的子类，已为二进制$ b $优化b个文件。

which is a subclass of StreamingHttpResponse optimized for binary files.

这篇关于提供大文件的Django Filewrapper内存错误，如何流式传输的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！