IIS，Helicon Zoo，Django - 文件上传网站 - 无法上传大于2.5MB（块问题） [英] IIS, Helicon Zoo, Django - File Upload Site - Unable to upload greater than 2.5MB (Chunk issue)

问题描述

在过去的几天里，我一直在不顾一切地思考这个问题，在我的具体问题上找不到很多帮助。

  Windows 7 
 IIS 7.5 
 Helicon Zoo 3.1.98.508 
 Django 1.6.3 
 Python 2.7.3 
  

我应该说我是Django和Helicon Zoo。

发生的事情是，虽然我似乎没有问题上传一个2.5MB的默认文件，我不能得到块正确地上传任何东西这个数额。虽然我确信我可以在设置文件中增加这个限制，但是我真的不想为了我的目的，我可能会看到需要上传大于我真正想要通过内存处理的文件。

当我尝试这个时，我所看到的是一个来自Helicon Zoo的关于一些丢失的标题的错误信息500.

 工作状态
进程已创建
 Windows错误
操作成功完成。 （错误代码：0）
内部模块错误
消息：HTTP头 - 预计
类型：ZooException 
文件：Jobs \JobFastCgi.cpp 
行： 391 
版本：3.1.98.508 
 STDERR 
空stderr 
  

起初，我想也许这是临时文件夹做的事情，所以我设置

  FILE_UPLOAD_TEMP_DIR = [os.path.join （BASE_DIR，'temp'）] 
  

在我的设置文件中，但似乎没有任何有用的东西。

然后我发现如何启用django的日志记录功能，真正开启了我的东西。我知道现在错误来自哪里..但我不知道我应该做些什么来解决它。

  [29 / Apr / 2014 11:37:00] ERROR [django.request：226]内部服务器错误：/ upload / 
 Traceback（最近一次调用最后一次）：
文件E： \mysite\venv\lib\site-packages\django\core\handlers\base.py，线路107，在get_response 
响应= middleware_method（请求，回调，callback_args，callback_kwargs ）
在process_view 
文件E：\mysite\venv\lib\site-packages\django\middleware\csrf.py，第170行，request_csrf_token = request.POST .get（'csrfmiddlewaretoken'，''）
文件E：\mysite\venv\lib\site-packages\django\core\handlers\wsgi.py，第146行，在_get_post 
 self._load_post_and_files（）
文件E：\mysite\venv\lib\site-packages\django\http\request.py，第215行，一世n _load_post_and_files 
 self._post，self._files = self.parse_file_upload（self.META，data）
文件E：\mysite\venv\lib\site-packages\django\\ \\\\\\\\\\\\\\\\\\\\\\\\'\\\\\\\\\\\\\\\\\\ \\\\\\\\\\\\\\\'\\\\\\\\\\\\\\\\\\\\\\\\'\\\\\\\\\\\\\\\\\\\文件\uploadhandler.py，第135行，在new_file中
 self.file = TemporaryUploadedFile（self.file_name，self.content_type，0，self.charset）
文件E：\mysite\在__init__ 
 dir = settings.FILE_UPLOAD_TEMP_DIR）
文件E：\\\\\\\\\\\\\\\\\\\\\\\\' mysite \venv\lib\site-packages\django\core\files\temp.py，第27行，在__init__ 
 dir = dir）
在mkstemp 
中返回文件C：\python27\lib\tempfile.py，第300行，返回_mkstemp_inner（dir，前缀，后缀，标志）
文件C：\python27\\ \\ lib \tempfile.py，第235行，在_mkstemp_inner 
 fd = _os.open（file，flags，0600）
 TypeError：强制为Unicode：需要字符串或缓冲区，找到列表
  

这是什么意思？我的块是以某种方式作为列表对象进来的，我没有意识到这一点？

这是我的应用程序代码，它包含在我的Blog应用程序中，我的主要应用程序，它似乎比创建一个应用程序所有它自己更简单。这是一个错误？
$ b

forms.py

  from django import forms 
 $ b $ class UploadFileForm（forms.Form）：
 title = forms.CharField（max_length = 50）
 file = forms.FileField（）
  

views.py

 从django.shortcuts导入渲染，选择render_to_response 
。从django.http进口从django.template进口的RequestContext的HttpResponse，HTTP404，HttpResponseRedirect 
，装载机
从blog.forms导入UploadFileForm 
 from blog.models import Blog $ b $ from blog.uploads import handle_uploaded_file 
 
 $ b def索引（请求）：
 latest_blog_list = Blog.objects.order_by（' -  pub_date'）[：5] 
 template = loader.get_template（'blog / index.html'）
 context = RequestContext（request，{$ b $''latest_blog_list '：latest_blog_list，
}）
 return HttpResponse（template.render（context））
 
 def （请求，blog_id）：
尝试：
 blog = Blog.objects.get（pk = blog_id）
除外：
 raise Http404 
 return render（request， 'blog / detail.html'，{'blog'：blog}）
 
 def upload_file（request）：$ b $ if request.method =='POST'：
 form = UploadFileForm（request.POST，request.FILES）
 if form.is_valid（）：
 handle_uploaded_file（request.FILES ['file']）
＃form.save（）
返回HttpResponseRedirect（'/ upload /'）
 else：
 form = UploadFileForm（）
返回render_to_response（'upload / upload.html'，{'form'：form}，context_instance = RequestContext （请求））
  

uploads.py

 进口口令
 
 saveDir ='E：\\uploads\\'
 
 def handle_uploaded_file（f）：
＃logging.debug（'upload_here'）
如果f：
 destination =打开（saveDir + f.name，'wb +'）
在f.chunks（）中为块：
 destination.write（块）
 destination.close（）
  

app urls.py



<$ p $从django.conf.urls导入模式，url
从博客导入视图

urlpatterns = patterns（''，
url（r'^ $'，views.index，name ='index'），
url（r'^（？P< blog_id> \d +）/ $'，views.detail，name ='detail'），
url（r'^ upload / $'，views.upload_file，name ='upload'），
）


project urls.py

  from django.conf.urls import patterns ，包括从django.conf导入设置的url 
从django.conf.urls.static导入设置
从django.contrib导入静态
 import admin $ b $ admin.autodiscover（）
 
 urlpatterns = patterns（''，
＃示例：
＃url （r'^ blog $'，'project.views.home'，name ='home'），
＃url（r'^ blog /'，include（'blog.urls'）），
 url（r'^ /？'，include（'blog.urls'，namespace =blog）），
 url（r'^ admin /'，include（admin.site.urls）），
 url（r'^ upload /'，include（'blog.urls'，namespace =upload）），
）
  

我也应该注意到，这些大文件的部分实际上出现在我的上传位置，但不超过1兆大小。就像这个过程在实际出于某种原因之前停下来一样。然后，我必须停止并重新启动IIS以使站点重新运行。

解决方案

所以有几个不同的问题这里。首先，我发现：

  TypeError：强制为Unicode：需要字符串或缓冲区，找到列表
 
 code> 

错误不是因为我的代码发送了对象，而是由于设置



  FILE_UPLOAD_TEMP_DIR = [os.path.join（BASE_DIR，'temp'）] 
  

需要转换为

pre $ FILE_UPLOAD_TEMP_DIR = os.path.join（BASE_DIR，' temp'）


这是什么被发送作为一个列表，当它不应该是简单的尝试复制django项目设置文件中的其他字段时出错。其次，我们应该注意到DIRS和DIR之间的区别，当然表示一个单引用。

---

其次，错误继续解决这个问题。经过几天的殴打，我试图找到一个IIS设置，可能会截断我的上传到1MB的大小，或者可能是一个Django的设置，我忽略了，甚至测试一个预制的应用程序，只是仍然击中了同样的砖墙，当我阅读Helicon Zoo错误报告的配置变量列表时，终于明白了我注意到的东西...

pre $ postBuffer = 1024


子之一...所以经过一些研究，我发现我可以修改这个设置

  POST_BUFFER =4096
 通过在我的Helicon Zoo web.config中使用环境变量我可以突然上传超过1MB的任何东西...任何超过4MB被截断，我得到一个错误！
 
  
 
 
更新：感谢rukeba上面的评论，他的解决方案是这个问题的最终答案，因为Twisted引擎没有这个问题。
I've been wracking my brain over this for the last few days now and haven't been able to find much in the way of help for my specific issue. Here's my setup.
Windows 7
IIS 7.5
Helicon Zoo 3.1.98.508
Django 1.6.3
Python 2.7.3
I should state that I'm new to Django and Helicon Zoo.

What's happening is that while I seem to have no issue uploading a file under the default of 2.5mb, I cannot get chunks to upload correctly for anything over this amount. While I'm sure I could up this limit in my settings file, I'd really rather not as for my purpose I could see needing to upload larger files than what I would really want being handled by memory.

What I'm seeing when I attempt this is a rather uninformative Error 500 from Helicon Zoo about some missing headers
Worker Status
The process was created
Windows error
The operation completed successfully. (ERROR CODE: 0)
Internal module error
message: HTTP-headers - are expected  
type: ZooException 
file: Jobs\JobFastCgi.cpp 
line: 391 
version: 3.1.98.508 
STDERR
Empty stderr
At first, I thought maybe it was something to do with the temp folder so I set
FILE_UPLOAD_TEMP_DIR = [os.path.join(BASE_DIR, 'temp')]
in my settings file, but it didn't seem to do anything useful.

I then discovered how to enable logging for django which really opened things up to me. I know now where the error is coming from.. but I have no clue what I'm supposed to do to get around it..
[29/Apr/2014 11:37:00] ERROR [django.request:226] Internal Server Error: /upload/
Traceback (most recent call last):
  File "E:\mysite\venv\lib\site-packages\django\core\handlers\base.py", line 107, in get_response
    response = middleware_method(request, callback, callback_args, callback_kwargs)
  File "E:\mysite\venv\lib\site-packages\django\middleware\csrf.py", line 170, in process_view
    request_csrf_token = request.POST.get('csrfmiddlewaretoken', '')
  File "E:\mysite\venv\lib\site-packages\django\core\handlers\wsgi.py", line 146, in _get_post
    self._load_post_and_files()
  File "E:\mysite\venv\lib\site-packages\django\http\request.py", line 215, in _load_post_and_files
    self._post, self._files = self.parse_file_upload(self.META, data)
  File "E:\mysite\venv\lib\site-packages\django\http\request.py", line 180, in parse_file_upload
    return parser.parse()
  File "E:\mysite\venv\lib\site-packages\django\http\multipartparser.py", line 197, in parse
    charset)
  File "E:\mysite\venv\lib\site-packages\django\core\files\uploadhandler.py", line 135, in new_file
    self.file = TemporaryUploadedFile(self.file_name, self.content_type, 0, self.charset)
  File "E:\mysite\venv\lib\site-packages\django\core\files\uploadedfile.py", line 61, in __init__
    dir=settings.FILE_UPLOAD_TEMP_DIR)
  File "E:\mysite\venv\lib\site-packages\django\core\files\temp.py", line 27, in __init__
    dir=dir)
  File "C:\python27\lib\tempfile.py", line 300, in mkstemp
    return _mkstemp_inner(dir, prefix, suffix, flags)
  File "C:\python27\lib\tempfile.py", line 235, in _mkstemp_inner
    fd = _os.open(file, flags, 0600)
TypeError: coercing to Unicode: need string or buffer, list found
What does this mean? Are my chunks coming in as a list object somehow and I don't realize it?

Here's my code for the app, it's included in my Blog app I made since it's my main app and it seemed simpler than creating an app all it's own. Was that a mistake?

forms.py
from django import forms

class UploadFileForm(forms.Form):
    title = forms.CharField(max_length=50)
    file = forms.FileField()
views.py
from django.shortcuts import render, render_to_response
from django.http import HttpResponse, Http404, HttpResponseRedirect
from django.template import RequestContext, loader
from blog.forms import UploadFileForm
from blog.models import Blog
from blog.uploads import handle_uploaded_file


def index(request):
    latest_blog_list = Blog.objects.order_by('-pub_date')[:5]
    template = loader.get_template('blog/index.html')
    context = RequestContext(request, {
        'latest_blog_list': latest_blog_list,
        })
    return HttpResponse(template.render(context))

def detail(request, blog_id):
    try:
        blog = Blog.objects.get(pk=blog_id)
    except:
        raise Http404
    return render(request, 'blog/detail.html', {'blog': blog})

def upload_file(request):
    if request.method == 'POST':
        form = UploadFileForm(request.POST, request.FILES)
        if form.is_valid():
            handle_uploaded_file(request.FILES['file'])
            #form.save()
            return HttpResponseRedirect('/upload/')
    else:
        form = UploadFileForm()
    return render_to_response('upload/upload.html', {'form': form}, context_instance=RequestContext(request))
uploads.py
import os

saveDir = 'E:\\uploads\\'

def handle_uploaded_file(f):
    #logging.debug('upload_here')
    if f:
        destination = open(saveDir + f.name, 'wb+')
        for chunk in f.chunks():
            destination.write(chunk)
        destination.close()
app urls.py
from django.conf.urls import patterns, url
from blog import views

urlpatterns = patterns('',
                       url(r'^$', views.index, name='index'),
                       url(r'^(?P<blog_id>\d+)/$', views.detail, name='detail'),
                       url(r'^upload/$', views.upload_file, name='upload'),
)
project urls.py
from django.conf.urls import patterns, include, url
from django.conf import settings
from django.conf.urls.static import static
from django.contrib import admin
admin.autodiscover()

urlpatterns = patterns('',
    # Examples:
    # url(r'^$', 'project.views.home', name='home'),
    # url(r'^blog/', include('blog.urls')),
    url(r'^/?', include('blog.urls', namespace="blog")),
    url(r'^admin/', include(admin.site.urls)),
    url(r'^upload/', include('blog.urls', namespace="upload")),
)
I should also note that portions of these large files are actually appearing in my upload location, but nothing more than 1 meg in size. It's like the process stops before it's actually done for some reason. I then have to stop and restart IIS to get the site to even function again.
解决方案
So there were a couple of different problems here. First off, I discovered that the
TypeError: coercing to Unicode: need string or buffer, list found
error was not because of the object being sent by my code, but rather due to the setting
FILE_UPLOAD_TEMP_DIR = [os.path.join(BASE_DIR, 'temp')]
needing to instead be
FILE_UPLOAD_TEMP_DIR = os.path.join(BASE_DIR, 'temp')
THAT is what was being sent as a list when it shouldn't have been, simple mistake in trying to replicate other fields within the django project settings file. I should have noticed the difference between DIRS and DIR, of course denoting a singular reference.



Secondly, the error continued after resolving that matter. After a couple days of beating my head trying to find maybe an IIS setting that could be truncating my uploads to 1MB in size, or potentially a django setting I was overlooking, and even testing a prebuilt app, only to still hit the same brick wall, it finally dawned on me when reading through the Helicon Zoo error report's list of configuration variables, that I noticed something...
postBuffer = 1024
Son of a... So after a little research I discovered I could modify this setting by using an environment variable inside of my Helicon Zoo web.config.
POST_BUFFER = "4096"
Viola.. I can suddenly upload anything over 1mb in size... anything over 4mb is truncated and I get an error!

So I finally found the issue, but its completely surprising to me that I can't find any sort of documentation regarding this as being an issue. Surely I'm not the only person whose come across this as it's a default setting! I wish I could remove it altogether as even being a limitation since django handles this just fine itself, I really don't need Helicon doing it for me. Unfortunately it looks like if I set the value to 0 then no uploads will work at all. So for now I'm going to set this value to something arbitrarily high just because frankly it's not needed and doesn't help me in any way.

UPDATE: Thanks to rukeba's comments above, his solution was the final answer to this issue as the Twisted engine does not have this problem.

                        
这篇关于IIS，Helicon Zoo，Django  - 文件上传网站 - 无法上传大于2.5MB（块问题）的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！