扩展蟒蛇/ Django的C和与Apache的mod_wsgi [英] Extend python/django with c and mod_wsgi with apache
问题描述
我有一个关于用C code和mod_wsgi的扩展Python的问题。
我在查询PostgreSQL数据库生成报告Apache服务器一个Django应用程序。在一些报告系统创建了结果的CSV文件。要创建这个CSV文件，有时系统必须处理两个以上的几十万与蟒蛇注册，这显然是非常缓慢的。为了加速这个问题，我们一个程序性I2C模块，做好这项工作，并且提高了速度多次。我们已经与ctypes的尝试和创建用C Python模块，与Apache和mod_wsgi的执行工作都在runserver，但崩溃罚款。
在的httpd-error.log中的错误是：
[周三7月27日2时33分52秒2011] [声明]儿童PID 44657退出信号分割故障（11）
¿任何建议？
在code是：
＃创建一个具有适当的CSV头中的Htt presponse对象。
响应= Htt的presponse（MIME类型='应用程序/ x-ZIP-COM pressed'）
响应['内容处置'] = \\
'附件;文件名='+文件名+.ZIPP0 ='descarga_'+ STR（datetime.today（））+'的.csv'P1 = settings.DATABASES ['默认'] ['名称']
P2 = settings.DATABASES ['默认'] ['USER']＃lib.generar（string_at（P0），P1，P2，string_at（STR（INIT）），string_at（STR（结束）），string_at（STR（供应商）））进口generador
generador.generar（P0，P1，P2，STR（INIT），STR（结束），STR（供应商））
据崩溃后，我们调用generador.generar（），它是用C开发的外部模块
我也使用gdb尝试作为@GrahamDumpleton suggets，这是输出，不是很有用：（
调试器开始，当我CLIC一种执行调用C模块，它属于阿恩在段错误的链接
（GDB）运行-X
正在调试的程序已经启动了。
从头开始呢？ （是或否）Y
启动程序：在/ usr / local / sbin中/ httpd的-X
[新LWP 101064]
[新主题28501140（LWP 101064）]
计划接收信号SIGSEGV，分割过错。
[切换主题28501140（LWP 101064）]
从/lib/libc.so.7 0x2847d423在FWRITE（）
在Segmetation故障后嘛，我试图在调试器中'，其中'命令，这是它给：
（GDB），其中
从/lib/libc.so.7＃0 0x2847d423在FWRITE（）
在generar＃1 0x293f8d09（）从/ruta/al/codigo/generador.so
＃2 0x28912caa从/usr/local/lib/libpython2.7.so PyCFunction_Call（）
＃3 0x2896e49a从/usr/local/lib/libpython2.7.so PyEval_EvalFrameEx（）
＃4，从/usr/local/lib/libpython2.7.so 0x2897044b在PyEval_Eval codeEX（）
＃5 0x288feafd从/usr/local/lib/libpython2.7.so PyClassMethod_New（）
＃6 0x288d622c从/usr/local/lib/libpython2.7.so PyObject_Call（）
＃7 0x2896c27a从/usr/local/lib/libpython2.7.so PyEval_EvalFrameEx（）
＃8 0x2896fadc从/usr/local/lib/libpython2.7.so PyEval_EvalFrameEx（）
＃9，从/usr/local/lib/libpython2.7.so 0x2897044b在PyEval_Eval codeEX（）
＃10从/usr/local/lib/libpython2.7.so PyClassMethod_New（）0x288fea0a
＃11从/usr/local/lib/libpython2.7.so PyObject_Call（）0x288d622c
＃12 0x288e4fd8在PyClass_IsSubclass（）从/usr/local/lib/libpython2.7.so
＃13从/usr/local/lib/libpython2.7.so PyObject_Call（）0x288d622c
＃14从/usr/local/lib/libpython2.7.so _PyObject_LookupSpecial（）0x2893044c
＃15从/usr/local/lib/libpython2.7.so PyObject_Call（）0x288d622c
从/usr/local/lib/libpython2.7.so＃16 0x28968ec4在PyEval_CallObjectWithKeywords（）
＃17 0x2889b229在Adapter_run（个体经营= 0x28b4dd58，对象= 0x28c7d50c）在mod_wsgi.c：3841
＃18 0x2889be50在mod_wsgi.c wsgi_execute_script（R = 0x28cb4058）：6547
＃19 0x2889de66在mod_wsgi.c wsgi_hook_handler（R = 0x28cb4058）：9080
＃20 0x08076b19在config.c ap_run_handler（R = 0x28cb4058）：157
＃21 0x08079dee在config.c ap_invoke_handler（R = 0x28cb4058）：376
＃22 0x08084eb0在http_request.c ap_process_request（R = 0x28cb4058）：282
＃23 0x0808201b在http_core.c ap_process_http_connection（C = 0x28b201f0）：190
＃24 0x0807de09在connection.c ap_run_process_connection（C = 0x28b201f0）：43
＃25 0x08089791在child_main（child_num_arg =变量child_num_arg不可用。
）在prefork.c：662
＃26 0x080899e3在prefork.c make_child（S = 0x28510f10，插槽= 0）：707
＃27 0x0808a591在prefork.c ap_mpm_run（_pconf = 0x2850f018，PLOG = 0x2853d018，S = 0x28510f10）：983
＃28 0x08064195主（ARGC = 676384792，argv的= 0x28b1e018）在main.c中：739
尝试设置：
WSGIApplicationGroup％{} GLOBAL
和强迫在主要的Python间preTER运行应用程序。
可能你的C扩展不能正常使用Python编写的分际preters工作。
请参阅：
的http://$c$c.google.com/p/modwsgi/wiki/ApplicationIssues#Python_Simplified_GIL_State_API
您code也可能仅仅是车下的mod_wsgi使用它显示出在哪里命令行Python做的问题。
请参阅：
的http://$c$c.google.com/p/modwsgi/wiki/DebuggingTechniques#Debugging_Crashes_With_GDB
如何调试那些碰撞发生。
更新1
在构建扩展模块，以帮助在调试GDB，确保优化未打开编译扩展模块和调试选项时。
有关setup.py驱动扩展模块构建，我觉得有必要补充一下：
从导入distutils的sysconfig中
虚拟= sysconfig.get_config_vars（'CFLAGS'，'OPT'）
config_vars = sysconfig._config_vars
config_vars ['CFLAGS'] = config_vars ['CFLAGS']。替换（-Os'，''）
config_vars ['OPT'] = config_vars ['OPT']。替换（-Os'，''）
这样，才能摆脱-Os选项。无论是-Os或-O将取决于你的Python安装。
检查然后在编译器选项-g的情况下扩展模块内置，并将其添加如果需要的话。
一旦做到这一点，那么你可以使用gdb来实际转储变量的值，得到实际的行号等。
I have a question about extending python with c code and mod_wsgi.
I have a django app in an apache server that query a postgresql database to generate reports. In some reports the system creates a csv file with the results. To create this csv file, some times the system must process more than two hundred thousands registers with python, that is obviously very slow. To accelerate this, we programed a c module that do this work, and that improves the speed many times. We have tried with ctypes and create a python module with c, both works fine in the runserver but crashes when executed with apache and mod_wsgi.
The error in httpd-error.log is:
[Wed Jul 27 02:33:52 2011] [notice] child pid 44657 exit signal Segmentation fault (11)
¿Any suggestion?
The code is:
# Creates the HttpResponse object with the appropriate CSV header.
response = HttpResponse(mimetype='application/x-zip-compressed')
response['Content-Disposition'] = \
'attachment; filename=' + filename + '.zip'
p0 = 'descarga_' + str(datetime.today()) + '.csv'
p1 = settings.DATABASES['default']['NAME']
p2 = settings.DATABASES['default']['USER']
#lib.generar(string_at(p0),p1,p2,string_at(str(init)),string_at(str(end)),string_at(str(provider)))
import generador
generador.generar(p0,p1,p2,str(init),str(end),str(provider))
It crashes after we call generador.generar() that is an external module developed in C
I have also tried with GDb as @GrahamDumpleton suggets and this is the output, not very useful :(
The debugger starts and when i clic the link that executes that calls the c module it falls agin in segmentation fault
(gdb) run -X
The program being debugged has been started already.
Start it from the beginning? (y or n) y
Starting program: /usr/local/sbin/httpd -X
[New LWP 101064]
[New Thread 28501140 (LWP 101064)]
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 28501140 (LWP 101064)]
0x2847d423 in fwrite () from /lib/libc.so.7
Well after the Segmetation fault, I tried with the 'where' command in the debugger and this is what it give:
(gdb) where
#0 0x2847d423 in fwrite () from /lib/libc.so.7
#1 0x293f8d09 in generar () from /ruta/al/codigo/generador.so
#2 0x28912caa in PyCFunction_Call () from /usr/local/lib/libpython2.7.so
#3 0x2896e49a in PyEval_EvalFrameEx () from /usr/local/lib/libpython2.7.so
#4 0x2897044b in PyEval_EvalCodeEx () from /usr/local/lib/libpython2.7.so
#5 0x288feafd in PyClassMethod_New () from /usr/local/lib/libpython2.7.so
#6 0x288d622c in PyObject_Call () from /usr/local/lib/libpython2.7.so
#7 0x2896c27a in PyEval_EvalFrameEx () from /usr/local/lib/libpython2.7.so
#8 0x2896fadc in PyEval_EvalFrameEx () from /usr/local/lib/libpython2.7.so
#9 0x2897044b in PyEval_EvalCodeEx () from /usr/local/lib/libpython2.7.so
#10 0x288fea0a in PyClassMethod_New () from /usr/local/lib/libpython2.7.so
#11 0x288d622c in PyObject_Call () from /usr/local/lib/libpython2.7.so
#12 0x288e4fd8 in PyClass_IsSubclass () from /usr/local/lib/libpython2.7.so
#13 0x288d622c in PyObject_Call () from /usr/local/lib/libpython2.7.so
#14 0x2893044c in _PyObject_LookupSpecial () from /usr/local/lib/libpython2.7.so
#15 0x288d622c in PyObject_Call () from /usr/local/lib/libpython2.7.so
#16 0x28968ec4 in PyEval_CallObjectWithKeywords () from /usr/local/lib/libpython2.7.so
#17 0x2889b229 in Adapter_run (self=0x28b4dd58, object=0x28c7d50c) at mod_wsgi.c:3841
#18 0x2889be50 in wsgi_execute_script (r=0x28cb4058) at mod_wsgi.c:6547
#19 0x2889de66 in wsgi_hook_handler (r=0x28cb4058) at mod_wsgi.c:9080
#20 0x08076b19 in ap_run_handler (r=0x28cb4058) at config.c:157
#21 0x08079dee in ap_invoke_handler (r=0x28cb4058) at config.c:376
#22 0x08084eb0 in ap_process_request (r=0x28cb4058) at http_request.c:282
#23 0x0808201b in ap_process_http_connection (c=0x28b201f0) at http_core.c:190
#24 0x0807de09 in ap_run_process_connection (c=0x28b201f0) at connection.c:43
#25 0x08089791 in child_main (child_num_arg=Variable "child_num_arg" is not available.
) at prefork.c:662
#26 0x080899e3 in make_child (s=0x28510f10, slot=0) at prefork.c:707
#27 0x0808a591 in ap_mpm_run (_pconf=0x2850f018, plog=0x2853d018, s=0x28510f10) at prefork.c:983
#28 0x08064195 in main (argc=676384792, argv=0x28b1e018) at main.c:739
Try setting:
WSGIApplicationGroup %{GLOBAL}
and forcing the application to run in main Python interpreter.
Possible your C extension isn't written properly to work with Python sub interpreters.
See:
http://code.google.com/p/modwsgi/wiki/ApplicationIssues#Python_Simplified_GIL_State_API
Your code could also just be buggy and using it under mod_wsgi shows out the problem where command line Python does.
See:
http://code.google.com/p/modwsgi/wiki/DebuggingTechniques#Debugging_Crashes_With_GDB
on how to debug where the crash is occurring.
UPDATE 1
When building your extension module, to aid debugging in gdb, ensure that optimisation isn't turned on when compiling extension module and that debug options are.
For setup.py driven extension module build, I find it necessary to add:
from distutils import sysconfig
dummy = sysconfig.get_config_vars('CFLAGS', 'OPT')
config_vars = sysconfig._config_vars
config_vars['CFLAGS'] = config_vars['CFLAGS'].replace(' -Os ', ' ')
config_vars['OPT'] = config_vars['OPT'].replace(' -Os ', ' ')
That way can get rid of -Os options. Whether it is -Os or -O will depend on your Python installation.
Check then for -g in compiler options when extension module built and add it if need be.
Once that is done you can then use gdb to actually dump out values of variables, get actual line numbers etc.
这篇关于扩展蟒蛇/ Django的C和与Apache的mod_wsgi的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
