简单的非ASCII字符串的Django头痛 [英] Django headache with simple non-ascii string
问题描述
我刚刚创建了以下模型:
I just created the following model:
class Categoria(models.Model):
nombre=models.CharField(max_length=30)
padre=models.ForeignKey('self', blank=True, null=True)
def __unicode__(self):
return self.nombre
然后注册到管理界面并同步数据库
Then registered to the admin interface and syncdb'd
如果我只添加纯 ASCII 字符，一切都好.但是，如果我添加一个名为á"的类别"(说点什么)，我会得到:
Everything ok if I just add plain ASCII chars. But if I add a "Categoria" named "á" (to say something) I get:
Environment:
Request Method: GET
Request URL: http://192.168.2.103:8000/administracion/locales/categoria/
Django Version: 1.1.1
Python Version: 2.6.4
Installed Applications:
['django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.admin',
'cruzandoelsuquiaDJ.locales']
Installed Middleware:
('django.middleware.common.CommonMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware')
Template error:
In template /usr/lib/pymodules/python2.6/django/contrib/admin/templates/admin/change_list.html, error at line 78
Caught an exception while rendering: ('ascii', 'xc3xa1', 0, 1, 'ordinal not in range(128)')
68 : {% endif %}
69 : {% endblock %}
70 :
71 : <form action="" method="post"{% if cl.formset.is_multipart %} enctype="multipart/form-data"{% endif %}>
72 : {% if cl.formset %}
73 : {{ cl.formset.management_form }}
74 : {% endif %}
75 :
76 : {% block result_list %}
77 : {% if action_form and actions_on_top and cl.full_result_count %}{% admin_actions %}{% endif %}
78 : {% result_list cl %}
79 : {% if action_form and actions_on_bottom and cl.full_result_count %}{% admin_actions %}{% endif %}
80 : {% endblock %}
81 : {% block pagination %}{% pagination cl %}{% endblock %}
82 : </form>
83 : </div>
84 : </div>
85 : {% endblock %}
86 :
Traceback:
File "/usr/lib/pymodules/python2.6/django/core/handlers/base.py" in get_response
92. response = callback(request, *callback_args, **callback_kwargs)
File "/usr/lib/pymodules/python2.6/django/contrib/admin/options.py" in wrapper
226. return self.admin_site.admin_view(view)(*args, **kwargs)
File "/usr/lib/pymodules/python2.6/django/views/decorators/cache.py" in _wrapped_view_func
44. response = view_func(request, *args, **kwargs)
File "/usr/lib/pymodules/python2.6/django/contrib/admin/sites.py" in inner
186. return view(request, *args, **kwargs)
File "/usr/lib/pymodules/python2.6/django/contrib/admin/options.py" in changelist_view
986. ], context, context_instance=context_instance)
File "/usr/lib/pymodules/python2.6/django/shortcuts/__init__.py" in render_to_response
20. return HttpResponse(loader.render_to_string(*args, **kwargs), **httpresponse_kwargs)
File "/usr/lib/pymodules/python2.6/django/template/loader.py" in render_to_string
108. return t.render(context_instance)
File "/usr/lib/pymodules/python2.6/django/template/__init__.py" in render
178. return self.nodelist.render(context)
File "/usr/lib/pymodules/python2.6/django/template/__init__.py" in render
779. bits.append(self.render_node(node, context))
File "/usr/lib/pymodules/python2.6/django/template/debug.py" in render_node
71. result = node.render(context)
File "/usr/lib/pymodules/python2.6/django/template/loader_tags.py" in render
97. return compiled_parent.render(context)
File "/usr/lib/pymodules/python2.6/django/template/__init__.py" in render
178. return self.nodelist.render(context)
File "/usr/lib/pymodules/python2.6/django/template/__init__.py" in render
779. bits.append(self.render_node(node, context))
File "/usr/lib/pymodules/python2.6/django/template/debug.py" in render_node
71. result = node.render(context)
File "/usr/lib/pymodules/python2.6/django/template/loader_tags.py" in render
97. return compiled_parent.render(context)
File "/usr/lib/pymodules/python2.6/django/template/__init__.py" in render
178. return self.nodelist.render(context)
File "/usr/lib/pymodules/python2.6/django/template/__init__.py" in render
779. bits.append(self.render_node(node, context))
File "/usr/lib/pymodules/python2.6/django/template/debug.py" in render_node
71. result = node.render(context)
File "/usr/lib/pymodules/python2.6/django/template/loader_tags.py" in render
24. result = self.nodelist.render(context)
File "/usr/lib/pymodules/python2.6/django/template/__init__.py" in render
779. bits.append(self.render_node(node, context))
File "/usr/lib/pymodules/python2.6/django/template/debug.py" in render_node
71. result = node.render(context)
File "/usr/lib/pymodules/python2.6/django/template/loader_tags.py" in render
24. result = self.nodelist.render(context)
File "/usr/lib/pymodules/python2.6/django/template/__init__.py" in render
779. bits.append(self.render_node(node, context))
File "/usr/lib/pymodules/python2.6/django/template/debug.py" in render_node
81. raise wrapped
Exception Type: TemplateSyntaxError at /administracion/locales/categoria/
Exception Value: Caught an exception while rendering: ('ascii', 'xc3xa1', 0, 1, 'ordinal not in range(128)')
我的 django 版本是 1.1，我的数据库是 5.1.37-1ubuntu5，带有 utf8 字符集，表使用的是 utf8_bin 排序规则.
My django version is 1.1 and my database is 5.1.37-1ubuntu5 with utf8 charset and the table is using a utf8_bin collation.
这个问题似乎太简单了，我是一个 django 新手，所以如果我错过了一些非常简单的东西，我很抱歉:)
This problem seems too basic to be true, and I'm a django newbie so I'm sorry in advance if I'm missing something very simple :)
推荐答案
Django 一般都有很好的 Unicode 支持(参见 Django 1.1 "Unicode data" 文档 了解详细信息).在我的代码中，我发现，如果我在使用简单的 Unicode 功能时遇到问题，问题通常是我没有很好地理解 Django 的细节，而不是 Django 在其 Unicode 支持方面存在错误.
Django generally has very good Unicode support (see the Django 1.1 "Unicode data" documentation for details). In my code I find that, if I'm having a problem with simple Unicode features, the problem usually is that I'm not understanding Django's details well, not that Django has a bug in its Unicode support.
Unicode 数据"页面告诉我们所有 Django 的数据库后端......自动将从数据库检索到的字符串转换为 Python Unicode 字符串.您甚至不需要告诉 Django 您的数据库使用什么编码:即透明地处理."所以你的简单 return self.nombre 应该返回一个 Python Unicode 字符串.
The "Unicode Data" page tells us that "All of Django’s database backends ... automatically convert strings retrieved from the database into Python Unicode strings. You don’t even need to tell Django what encoding your database uses: that is handled transparently." So your simple return self.nombre should return a Python Unicode string.
但是，Django 1.1 "Databases" 页面 有一个重要说明关于 MySQL 后端如何处理 utf8_bin 排序规则:
However, the Django 1.1 "Databases" page has an important note about how the MySQL backend handles the utf8_bin collation:
...如果你真的想要区分大小写对特定列的比较或表，您将更改列或表使用 utf8_bin 排序规则.在这方面要注意的主要事项情况是，如果您使用的是 MySQLdb1.2.2，Django中的数据库后端然后将返回字节串(而不是unicode 字符串)对于任何字符它返回的字段从数据库.这是一个强烈的变化来自 Django 的正常实践总是返回 unicode 字符串.它由你，开发者来处理您将收到的事实字节串，如果你配置你的表使用 utf8_bin 排序规则.Django 本身应该可以顺利运行有这样的列，但如果你的代码必须准备好打电话django.utils.encoding.smart_unicode()有时如果它真的想工作具有一致的数据...
...if you really want case-sensitive comparisons on a particular column or table, you would change the column or table to use the utf8_bin collation. The main thing to be aware of in this case is that if you are using MySQLdb 1.2.2, the database backend in Django will then return bytestrings (instead of unicode strings) for any character fields it returns receive from the database. This is a strong variation from Django's normal practice of always returning unicode strings. It is up to you, the developer, to handle the fact that you will receive bytestrings if you configure your table(s) to use utf8_bin collation. Django itself should work smoothly with such columns, but if your code must be prepared to call django.utils.encoding.smart_unicode() at times if it really wants to work with consistent data ...
因此，在您的原始示例中，nombre"列使用了 utf8_bin 排序规则.这意味着 self.nombre 正在返回一个 Python 字节字符串.当您将它放在需要 Python Unicode 字符串的表达式中时，Python 会执行其默认转换.这相当于 self.nombre.decode('ascii').当然，.decode('ascii') 在遇到任何高于 0x7F 的字节时都会失败，例如编码á"的 UTF-8 字节.
So, in your original example, the column "nombre" used utf8_bin collation. This meant that self.nombre was returning a Python byte string. When you put it in an expression that required a Python Unicode string, Python performed its default conversion. This is the equivalent of self.nombre.decode('ascii'). And of course, .decode('ascii') fails when it encounters any byte above 0x7F, such as the UTF-8 bytes which encode "á".
您发现了解决此问题的两种方法.第一种是将 self.nombre 返回的 Python 字节字符串显式转换为 Python Unicode 字符串.我敢打赌以下更简单的代码会起作用:
You discovered the two ways to solve this problem. The first is to convert the Python byte string returned by self.nombre into a Python Unicode string explicitly. I'll bet the following simpler code would have worked:
return self.nombre.decode('utf8')
第二种方法是更改nombre"列的 MySQL 排序规则，这会导致 Django 的 MySQL 后端返回 Python Unicode 字符串而不是异常字节字符串.然后你的原始表达式给出了一个 Python Unicode 字符串:
The second approach is to change the MySQL collation for column "nombre", which causes Django's MySQL backend to return Python Unicode strings instead of the unusual byte strings. Then your original expression gives a Python Unicode string:
return self.nombre
希望这会有所帮助.
这篇关于简单的非ASCII字符串的Django头痛的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
