带有生成器的WSGI文件流 [英] WSGI file streaming with a generator

问题描述

我有以下代码:

def application(env, start_response):
    path = process(env)
    fh = open(path,'r')
    start_response('200 OK', [('Content-Type','application/octet-stream')])
    return fbuffer(fh,10000)


def fbuffer(f, chunk_size):
    '''Generator to buffer file chunks'''  
    while True:
        chunk = f.read(chunk_size)      
        if not chunk: break
        yield chunk

我不确定这是正确的，但是我在互联网上发现的信息残片使我认为它应该起作用.基本上，我想分块地流式传输文件，并为此从我的应用程序函数中传回一个生成器.但是，这只会打印出标题，而实际上并没有发送回任何数据，任何人都可以告诉我为什么吗?

I'm not sure that it's right but the scraps of information I've found on the internet have led me to think it ought to work. Basically I want to stream a file out in chunks, and to do that I'm passing a generator back from my application function. However this only prints out the headers and doesn't actually send back any data, can anyone tell me why this is?

或者，如果这是完全错误的，那么执行此操作的最佳方法是什么?我无法将整个文件缓冲在内存中，因为我将要使用的文件可能是千兆字节.

Alternatively, if this is completely wrong, what's the best way to do this? I can't buffer the whole file in memory as the files I'll be working with are potentially gigabytes large.

第三级问题:完成输出后，关闭文件的最佳方法是什么?在我发布的代码中，我仍然看不到要关闭文件.

Tertiary question: What's the best way to close the file once I'm done outputting it? In the code I posted I can't see anyway to actually close the file.

(我正在使用uWSGI 1.2.4运行python 3.2.3)

(I'm running python 3.2.3 with uWSGI 1.2.4)

推荐答案

不小心，uwsgi注意不要让错误泄漏，但是如果您在更严格的实现中运行应用程序，如果将python提供的代码称为wsgiref.simple_server，则可以更轻松地看到问题所在.

Without some care, uwsgi is careful not to allow errors to leak, but a if you run your application in a stricter implementation, say the one provided with python as wsgiref.simple_server, you can more easily see the problem.

Serving <function application at 0xb65848> http://0.0.0.0:8000
Traceback (most recent call last):
  File "/usr/lib64/python3.2/wsgiref/handlers.py", line 138, in run
    self.finish_response()
  File "/usr/lib64/python3.2/wsgiref/handlers.py", line 179, in finish_response
    self.write(data)
  File "/usr/lib64/python3.2/wsgiref/handlers.py", line 264, in write
    "write() argument must be a bytes instance"
AssertionError: write() argument must be a bytes instance
localhost.localdomain - - [04/Aug/2012 16:27:08] "GET / HTTP/1.1" 500 59

问题在于，wsgi要求必须将往返于HTTP网关的数据用作bytes，但是当您使用open(path, 'r')时，python 3方便地将读取的数据转换为unicode，在python 3中是str，使用默认编码.

The problem is that wsgi requires that data transmitted to and from the HTTP gateway must be served as bytes, but when you use open(path, 'r'), python 3 conveniently converts the data read to unicode, what in python 3 is str, using the default encoding.

更改

fh = open(path, 'r')

到

fh = open(path, 'rb')
#                 ^

修复它.

这篇关于带有生成器的WSGI文件流的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！