uWSGI用于上传和处理文件 [英] uWSGI for uploading and processing files

问题描述

我有一个用Bottlepy编写的python Web应用程序.其唯一目的是允许人们上载将要处理的大文件(处理大约需要10-15分钟).

I have a python web application written in bottlepy. Its only purpose is to allow people to upload large files that will be processed (takes approximately 10-15 minutes to process).

上传代码相当简单:

@route('/upload', method='POST')
def upload_file():
  uploadfile = request.files.get('fileToUpload')
  if not uploadfile:
    abort(500, 'No file selected for upload')

  name,ext = os.path.splitext(uploadfile.filename)

  if ext not in ['.zip','.gz']:
    abort(500, 'File extension not allowed')

  try:
    uploadfile.save('./files')

    process_file(uploadfile.filename) #this function is not yet implemented

    return "uploaded file '%s' for processing" % uploadfile.filename
  except IOError as e:
    abort(409, "File already exists.")

我计划使用uWSGI部署此应用程序(但是，如果出于其他目的而更好地使用其他技术，则它并不是一成不变的.

I plan to deploy this application using uWSGI (however, if other technology is better for the purpose its not set in stone.

因此，对于将uWSGI用于此目的，我有一些疑问:

Because of this I have some questions regarding the use of uWSGI for such a purpose:

1. 如果文件上传需要几分钟，那么uWSGI如何能够在不阻塞的情况下处理其他客户端?
2. 是否可以使用uWSGI中的内置功能来卸载处理，以便用户在上载后获得响应并可以查询处理状态?

谢谢您的帮助.

推荐答案

如果文件上传需要几分钟，那么uWSGI将如何 处理其他客户而不受到阻碍?

If the file upload takes minutes, how will uWSGI be capable of handling other clients without blocking?

它将阻止. 一种解决方案是将像NGINX这样的Web服务器放在uWSGI之前，该服务器预先缓冲POST请求.因此，文件上传实际上将绑定到NGINX处理程序，直到完成，然后然后传递给uWSGI处理程序.

It will block. A solution is to put a webserver like NGINX in front of uWSGI that pre-buffers the POST request. So the file upload will be actually bound to an NGINX handler until is completed and then passed to the uWSGI handler.

有什么方法可以使用内置的方法来卸载处理 uWSGI中的功能，以便用户在上传后获得响应 并可以查询处理状态吗?

Is there any way the processing can be offloaded using built in functionality in uWSGI so that the user get a response after upload and can query for processing status?

您需要创建一个任务队列系统，以从Web处理程序中分流处理. 这是常见的最佳做法.随便看看python task queues. 对于内置功能，它实际上取决于您需要卸载的任务. 您可以使用内置的 uWSGI后台打印程序，或uWSGI les子. 这些是典型任务队列(例如非常著名的 Celery )的非常好的替代方案，但有局限性. 只需在您的情况下自己尝试即可.

You need to create a task queue system to offload the processing from the web handler. This is a common best practice. Just look around for python task queues. For builtin functionalities it really depends on the task you need to offload. You can use the builtin uWSGI spooler, or the uWSGI mules. These are very good alternatives to a typical task queue (like the very famous Celery) but have limitations. Just try it yourself in your scenario.

这篇关于uWSGI用于上传和处理文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！