django.db.utils.OperationalError:解析器堆栈溢出 [英] django.db.utils.OperationalError: parser stack overflow
问题描述
我有一个可重复使用的django应用程序，该应用程序应支持 python2.7 ， python 3.x 和 pypy .我在一开始就使用 python 2.7 开发了它，并且我的所有测试都运行良好.我也使它们也可以在 python3.3 中工作.但是我对 python3.4 ， pypy ， pypy3 有问题；
I have a re-usable django application which should support python2.7, python 3.x and pypy. I developed it in python 2.7 at the beginning and all of my tests are worked very well. I also made them worked in python3.3 too. But I have a problem with python3.4, pypy, pypy3;
django.db.utils.OperationalError: parser stack overflow
我的测试在 sqlite3 上运行.我检查了踪迹，我只能猜到它与查询大小有关.我找不到解决此问题的任何解决方案.
My tests run on sqlite3. I check the trace, I just could guess that it is about query size. I couldn't find any solution to solve this problem.
我在我的 python3.4 环境中覆盖了内置的 sqlite3 代码，以推测出哪个sql，会引起错误.这真的是很大的sql.这没关系，您不需要检查sql，我只是将其发布在此处以显示它的大小.这也可以更大.因为 queryset 是在for循环中的运行时构建的.
I overrided builtin sqlite3 code in my python3.4 environment to pring which sql, does it raise the error for. It is really big sql. It really doesn't matter, you don't need to check the sql, I just post it in here to show how it is big. This can also be bigger than that. Because queryset is building on runtime in for loop.
顺便说一句，就像我之前说的， python2.7 和 python3.3 都没有问题.问题给别人带来了.
By the way, as I told it before, There is no problem with python2.7 and python3.3. Problem raises for others.
有任何配置可以处理吗?
Are there any configuration to handle this?
这是sql:
SELECT "river_approvement".
"id", "river_approvement".
"content_type_id", "river_approvement".
"object_id", "river_approvement".
"field", "river_approvement".
"meta_id", "river_approvement".
"transactioner_id", "river_approvement".
"transaction_date", "river_approvement".
"status", "river_approvement".
"skip", "river_approvement".
"order", "river_approvement".
"enabled"
FROM "river_approvement"
INNER JOIN "river_approvementmeta"
ON("river_approvement".
"meta_id" = "river_approvementmeta".
"id") INNER JOIN "river_transition"
ON("river_approvementmeta".
"transition_id" = "river_transition".
"id") WHERE("river_approvement".
"field" = ? AND "river_transition".
"source_state_id"
IN(SELECT AB0.
"id"
FROM "river_state"
AB0 WHERE AB0.
"id"
IN(SELECT AA2.
"destination_state_id"
FROM "river_approvement"
AA0 INNER JOIN "river_approvementmeta"
AA1 ON(AA0.
"meta_id" = AA1.
"id") INNER JOIN "river_transition"
AA2 ON(AA1.
"transition_id" = AA2.
"id") WHERE(AA0.
"field" = ? AND AA2.
"source_state_id"
IN(SELECT Z0.
"id"
FROM "river_state"
Z0 WHERE Z0.
"id"
IN(SELECT Y2.
"destination_state_id"
FROM "river_approvement"
Y0 INNER JOIN "river_approvementmeta"
Y1 ON(Y0.
"meta_id" = Y1.
"id") INNER JOIN "river_transition"
Y2 ON(Y1.
"transition_id" = Y2.
"id") WHERE(Y0.
"field" = ? AND Y2.
"source_state_id"
IN(SELECT X0.
"id"
FROM "river_state"
X0 WHERE X0.
"id"
IN(SELECT W2.
"destination_state_id"
FROM "river_approvement"
W0 INNER JOIN "river_approvementmeta"
W1 ON(W0.
"meta_id" = W1.
"id") INNER JOIN "river_transition"
W2 ON(W1.
"transition_id" = W2.
"id") WHERE(W0.
"field" = ? AND W2.
"source_state_id"
IN(SELECT V0.
"id"
FROM "river_state"
V0 WHERE V0.
"id"
IN(SELECT U2.
"destination_state_id"
FROM "river_approvement"
U0 INNER JOIN "river_approvementmeta"
U1 ON(U0.
"meta_id" = U1.
"id") INNER JOIN "river_transition"
U2 ON(U1.
"transition_id" = U2.
"id") WHERE(U0.
"field" = ? AND U2.
"source_state_id"
IN( ? ) AND U0.
"object_id" = ? AND U0.
"content_type_id" = ? ))) AND W0.
"object_id" = ? AND W0.
"content_type_id" = ? ))) AND Y0.
"object_id" = ? AND Y0.
"content_type_id" = ? ))) AND AA0.
"object_id" = ? AND AA0.
"content_type_id" = ? ))) AND "river_approvement".
"object_id" = ? AND "river_approvement".
"content_type_id" = ? )
推荐答案
默认 sqlite3解析器堆栈大小是100个词汇项目.他们认为这很可能超出任何人的理解能力" .我在您的示例中看到许多嵌套级别:15个括号，9个"SELECT"，9个"WHERE"，9个"IN"，5个"AND".这是我可以用任何可能的解析器语法在堆栈上想象的必要条件的最小值.也许也加入了，或者将一些看不见的东西也算进去了，但是大小100是有意义的.可以使用YYSTACKDEPTH = -1或较大的正数(-1是一个动态深度堆栈，从100开始并每次加倍)重新编译Sqlite3，但这不是可重用的公共应用程序的解决方案.
The default sqlite3 parser stack size is 100 lexical items. They think "it is likely to be well beyond the ability of any human to comprehend". I see many nested levels in your example: 15 parenheses, 9 "SELECT", 9 "WHERE", 9 "IN", 5 "AND". That is the minimum of necessary terms that I can imagine on the stack with any possible parser gramatics. Maybe also joins or something invisible is counted into, but the size 100 makes sense. Sqlite3 can be recompiled with the option YYSTACKDEPTH=-1 or a big positive number (-1 is a dynamic depth stack starting with 100 and doubled everytimes), but it is not a solution for a reusable public application.
可以通过将复杂的术语(子选择)移到 AND 作为布尔表达式的第一项来进行一些改进.可以通过以下方式完成:
It can be improved a little by moving complicated terms (sub-select) before AND as the first term of the boolean expression. It can be done by:
MyModel.objects. \
filter(meta__transition__destination_state_id__in=
MyModel.objects.filter(...)
).filter(field=...)
在100个项目中，它实际上可以将堆栈大小提高5"AND".
It can improve the stack size really a little by 5 "AND" in 100 items.
您可能会在一个过滤器中使用具有更多条件的过滤器表达式: filter(field_1 = value_1，field_2 = value_2)与 filter(** {'field_1':value_1，"field_2":value_2}).字典中各项的顺序取决于相应CPython版本中 hash 函数的实现，甚至取决于字典本身在pypy上的实现细节.这就是为什么仅在某些Python版本中可能引发异常的原因.如果您添加了类似复杂的附加子选择，则每个Python都必须引发异常.
You probaby use a filter expression with more conditions in one filter: filter(field_1=value_1, field_2=value_2) which is the same as filter(**{'field_1': value_1, 'field_2': value_2}). The order of items in the dictionary depends on the implementation of hash function in the respective CPython version or even on the implemetation details of dictionary itself on pypy. This is why the exception may be raised only with some Python versions. If you add a similarly complicated additional subselect, every Python must raise the exception.
无需运行即可轻松检查Django queryset使用的结果SQL:
The result SQL used by Django queryset can be easily examined without running it:
print(my_complicated_queryset.query.get_compiler('default').as_sql())
'default'是 settings.DATABASES 中的连接别名.
这篇关于django.db.utils.OperationalError:解析器堆栈溢出的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
