PyODBC Cursor.fetchall() 导致 python 崩溃(segfault) [英] PyODBC Cursor.fetchall() causes python to crash (segfault)
问题描述
我在 Windows XP 上使用 Python 2.7.
I am using Python 2.7 on Windows XP.
我有一个简单的 Python 脚本，它按计划使用 pyodbc 从 AR 数据库中抓取数据，该数据库一直运行到今天.一旦光标到达特定行，我就会得到一个段错误.我在 C++ 中有类似的代码，检索结果没有问题，所以我认为这是 pyodbc 的问题.无论哪种方式，我都想抓住"这个错误.我尝试使用 subprocess 模块，但它似乎不起作用，因为一旦脚本遇到段错误，它就会挂在python.exe 遇到问题需要关闭"上.信息.我想我可以设置一些任意的时间框架来完成它，如果没有，则强制关闭该进程，但这似乎有点蹩脚.
I have a simple python script on a schedule that uses pyodbc to grab data from an AR database which has worked perfectly until today. I get a segfault once the cursor reaches a particular row. I have similar code in C++ which has no problem retrieving the results, so I figure this is an issue with pyodbc. Either way, I'd like to "catch" this error. I've tried to use the subprocess module, but it doesn't seem to work since once the script hits a segfault it just hangs on the "python.exe has encountered a problem and needs to close." message. I guess I could set some arbitrary time frame for it to complete in and, if it doesn't, force close the process, but that seems kind of lame.
我也在这里报告了这个问题 - http://code.google.com/p/pyodbc/issues/detail?id=278
I have reported the issue here as well - http://code.google.com/p/pyodbc/issues/detail?id=278
@paulsm4 - 我已经在下面回答了您的问题，谢谢！
@paulsm4 - I have answered your questions below, thanks!
问:您使用的是 Windows/XP(我想是 32 位)、Python 2.7 和 BMC补救措施.正确吗?
Q: You're on Windows/XP (32-bit, I imagine), Python 2.7, and BMC Remedy AR. Correct?
A:是的，它在 Win XP 32 位和 Win Server 2008 R2 64 位上失败.
A: Yes, it fails on Win XP 32 bit and Win Server 2008 R2 64 bit.
问:您(或者您的客户，如果他们购买了Remedy AR) 可以与 BMC 建立支持电话吗?
Q: Is there any chance you (or perhaps your client, if they purchased Remedy AR) can open a support call with BMC?
A:可能不会...
问:你能找出导致段错误的列吗?什么是不同"发生段错误时?
Q: Can you isolate which column causes the segfault? "What's different" when the segfault occurs?
A:就在这一行……但我现在已经根据您的建议将问题隔离开来.我使用循环来获取每个字段，直到发生段错误.
A: Just this particular row...but I have now isolated the issue with your suggestions below. I used a loop to fetch each field until a segfault occurred.
cursor.columns(table="mytable")
result = cursor.fetchall()
columns = [x[3] for x in result]
for x in columns:
print x
cursor.execute("""select "{0}"
from "mytable"
where id = 'abc123'""".format(x))
cursor.fetchall()
一旦我确定了导致段错误的列，我就尝试查询除该列之外的所有列，果然没有问题.
Once I identified the column that causes the segfault I tried a query for all columns EXCEPT that one and sure enough it worked no problem.
该列的数据类型为 CHAR(1024).我使用 C++ 来获取数据，并注意到该行的列在任何其他行中拥有最多的字符......1023！认为 PyODBC 的 C 代码中可能有一个缓冲区正在写入超出其边界的内容.
The column's data type was CHAR(1024). I used C++ to grab the data and noticed that the column for that row had the most characters in it out of any other row...1023! Thinking that maybe there is a buffer in the C code for PyODBC that is getting written to beyond its boundaries.
2) 启用 ODBC 跟踪:http://support.microsoft.com/kb/274551
2) Enable ODBC tracing: http://support.microsoft.com/kb/274551
3) 回发结果(包括失败的日志跟踪)
3) Post back the results (including the log trace of the failure)
好的，我已经用 ODBC 跟踪的结果创建了一个 pastebin - http://pastebin.com/6gt95rB8.为了保护无辜，我屏蔽了一些字符串值.
Ok, I have created a pastebin with the results of the ODBC trace - http://pastebin.com/6gt95rB8. To protect the innocent, I have masked some string values.
看起来可能是由于数据截断.
Looks like it may have been due to data truncation.
这是否为我们提供了有关如何解决问题的足够信息?我认为这是 PyODBC 中的一个错误，因为直接使用 C ODBC API 可以正常工作.
Does this give us enough info as to how to fix the issue? I'm thinking it's a bug within PyODBC since using the C ODBC API directly works fine.
更新
所以我编译了 PyODBC 进行调试，我收到一条有趣的消息 -
So I compiled PyODBC for debugging and I got an interesting message -
Run-Time Check Failure #2 - Stack around the variable 'tempBuffer' was corrupted.
虽然目前没看懂，但是调用栈如下——
While I don't currently understand it, the call stack is as follows -
pyodbc.pyd!GetDataString(Cursor * cur=0x00e47100, int iCol=0) Line 410 + 0xf bytes C++
pyodbc.pyd!GetData(Cursor * cur=0x00e47100, int iCol=0) Line 697 + 0xd bytes C++
pyodbc.pyd!Cursor_fetch(Cursor * cur=0x00e47100) Line 1032 + 0xd bytes C++
pyodbc.pyd!Cursor_fetchlist(Cursor * cur=0x00e47100, int max=-1) Line 1063 + 0x9 bytes C++
pyodbc.pyd!Cursor_fetchall(_object * self=0x00e47100, _object * args=0x00000000) Line 1142 + 0xb bytes C++
已解决！
通过确保缓冲区有足够的空间解决了问题.
Problem was solved by ensuring that the buffer had enough space.
在第 330 行的 getdata.cpp 中
In getdata.cpp on line 330
char tempBuffer[1024];
改为
char tempBuffer[1025];
编译并替换了站点包中旧的 pyodbc.pyd 文件，我们都很好！
Compiled and replaced the old pyodbc.pyd file in site-packages and we're all good!
感谢您的帮助！
推荐答案
问:您使用的是 Windows/XP(我想是 32 位)，Python 2.7 和 BMC Remedy AR.正确吗?
Q: You're on Windows/XP (32-bit, I imagine), Python 2.7, and BMC Remedy AR. Correct?
问:您(或者您的客户，如果他们购买了 Remedy AR)是否有机会与 BMC 打通支持电话?
Q: Is there any chance you (or perhaps your client, if they purchased Remedy AR) can open a support call with BMC?
问:你能找出导致段错误的列吗?发生段错误时有什么不同"?
Q: Can you isolate which column causes the segfault? "What's different" when the segfault occurs?
请执行以下操作:
1) 使用 Python/ODBC 尝试不同的select a,b,c"语句，看看您是否可以重现问题(独立于您的程序)并隔离特定列(或者，理想情况下，特定列和行！)
1) Try different "select a,b,c" statements using Python/ODBC to see if you can reproduce the problem (independent of your program) and isolate a specific column (or, ideally, a specific column and row!)
2) 启用 ODBC 跟踪:http://support.microsoft.com/kb/274551
2) Enable ODBC tracing: http://support.microsoft.com/kb/274551
3) 回发结果(包括失败的日志跟踪)
3) Post back the results (including the log trace of the failure)
4) 如果这不起作用 - 如果您不能让 BMC 技术支持参与 - 那么 B 计划可能是在 ODBC 库级别进行调试:
4) If that doesn't work - and if you can't get BMC Technical Support involved - then Plan B might be to debug at the ODBC library level:
问:哪种 C/C++ 编译器最适合您?
Q: What C/C++ compiler would work best for you?
这篇关于PyODBC Cursor.fetchall() 导致 python 崩溃(segfault)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
