Unicode问题与makemessages - 所有Django 1.6.2 Python 3.3 [英] Unicode issue with makemessages --all Django 1.6.2 Python 3.3

问题描述

从 Python 2.7升级项目 - > 3.3.1 和 Django 1.4 - > 1.6.2

Upgrading a project from Python 2.7 -> 3.3.1 and Django 1.4 -> 1.6.2.

更新代码后，我们的应用程序再次运行（在py3中）。

正在从 .mo 文件。

After updating code our application is running again (in py3).
Translations are being pulled from .mo files.

唯一的问题是我们旧的 .po 文件不能与

The only issue is that our old .po files cannot be used with

django-admin.py makemessages -a

它显示可爱

UnicodeDecodeError：'ascii'编解码器无法解码字节...

我们可以第一次运行 makemessages 并获取骨架文件。一旦我们将非ASCII（ǹ，è等）的任何翻译添加到 msgstr 值， makemessages 无法完成。

（如果我们运行具有更高冗长度模板的任何非ASCII字符的makemessages将被跳过。）

We can run makemessages for the first time and get the skeleton files. Once we add any translations that are non-ASCII (ǹ, è, etc.) to the msgstr values, makemessages fails to complete.
(If we run makemessages with higher verbosity templates with any NON-ASCII characters are skipped.)

我发现类似问题的错误报告，但是他们回到了1.3.x版本，但是对于上述版本没有任何帮助。

I've found bug reports for a similar issue, but they're back in the 1.3.x versions, but nothing really for the above versions.

更新，更多信息：

以下是发生例外情况：

../ python3.3 / subprocess.py line 847

Here's where the exception is happening:
../python3.3/subprocess.py line 847

def _translate_newlines(self, data, encoding):
    data = data.decode(encoding)
    return data.replace("\r\n", "\n").replace("\r", "\n")

encoding value is ANSI_X3.4-1968 。我已经将模板文件保存为UTF-8，以及.po文件。

The encoding value is ANSI_X3.4-1968. I've saved the template files as UTF-8, along with the .po file(s).

这里是.po标头（只是从makemessages自动创建的框架）：

Here's is the .po header (just the skeleton created automatically from makemessages):

# SOME DESCRIPTIVE TITLE.
# Copyright (C) YEAR THE PACKAGE'S COPYRIGHT HOLDER
# This file is distributed under the same license as the PACKAGE package.
# FIRST AUTHOR <EMAIL@ADDRESS>, YEAR.
#
#, fuzzy
msgid ""
msgstr ""
"Project-Id-Version: PACKAGE VERSION\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2014-02-28 22:42+0000\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language-Team: LANGUAGE <[e-mail]>\n"
"Language: \n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Plural-Forms: nplurals=2; plural=(n != 1);\n"  

以前在Python 2.7和Django 1.5之前（相同的文件）工作。

This worked before (same files) under Python 2.7 and Django 1.5

更新＃2

• 列表项目


• 创建一个新的裸体项目（django-admin.py startproject blah）


• 启用i18n等。


• 创建一个单一的翻译（仅在settings.py中）


• python3管理（＃python3），python3管理。 py makemessages -a）失败

• List item
• Created a new naked project (django-admin.py startproject blah)
• Enabled i18n, etc.
• Created a single translation (just in settings.py)
• Ran `makemessages -l de
• Py2.7 (#python manage.py makemessages -a) works as expected
• Py3.3 (#python3 manage.py makemessages -a) fails

可能会提交错误，将更新。

Possibly filing a bug, will update.

推荐答案

这样一来，Py2一切都很好，但Py3不好，所以我假设这是问题所在。

This threw me off as everything was fine with Py2, but not Py3, so I assumed that's where the problem would be.

问题在于部分原因是因为我正在使用Docker，并且在容器内运行 makemessages ，这些容器没有设置任何特定于bash的区域设置。

The issue was in part because I'm using Docker, and running makemessages from within the container which didn't have the locale set to anything in particular for bash.

我尝试的东西：

• 使用UTF-8保存文件 （有和没有BOM）


• 确保在 UTF-8 > .po 文件


• 创建一个新的空白石板项目


• 重新创建所有。 po 使用Py3 的文件（最初由Py2创建）

• Saving files with UTF-8 (with and without BOM)
• Ensuring I had UTF-8 in the header of the .po files
• Creating a new blank slate project
• Re-creating all .po files using Py3 (as they were originally created with Py2)

在这行847上的subprocess.py中抛出了最大的例外：

The top exception was being thrown in subprocess.py on this line 847:

def _translate_newlines(self, data, encoding):
    data = data.decode(encoding)
    return data.replace("\r\n", "\n").replace("\r", "\n")

传入的编码是 ANSI_X3.4-1968 ，这是奇怪的，因为我保存的文件为UTF-8等。（它被设置为ANSI ...由于我的bash会话没有专门设置的区域设置）。

The encoding being passed in was ANSI_X3.4-1968, which was weird as I had saved the files as UTF-8 etc (it was being set as ANSI... due to my bash session not have a locale set specifically).

答案

在我的Docker容器中，我没有在终端设置区域设置，所以他们是： p>

Answer
In my Docker container I had no locale settings set in the terminal, so they were:

# locale
LANG=
LANGUAGE=
LC_CTYPE="POSIX"
LC_NUMERIC="POSIX"
LC_TIME="POSIX"
LC_COLLATE="POSIX"
LC_MONETARY="POSIX"
LC_MESSAGES="POSIX"
LC_PAPER="POSIX"
LC_NAME="POSIX"
LC_ADDRESS="POSIX"
LC_TELEPHONE="POSIX"
LC_MEASUREMENT="POSIX"
LC_IDENTIFICATION="POSIX"
LC_ALL=

这些是我可用的区域设置（缺少我的特定区域， en_US .UTF-8 ，但只要是UTF-8就可以了）：

These were my available locales (missing my specific locale, en_US.UTF-8, but as long as it's UTF-8 I'm okay):

# locale -a
C
C.UTF-8
POSIX

将其放在〜/ .bashrc 中：

export LC_ALL=C.UTF-8
export LANG=C.UTF-8
export LANGUAGE=C.UTF-8  

现在我得到 UTF-8 e内容类型在 subprocess.py 中，一切都适用于Py3 / Django1.6 =）

And now I get UTF-8 as the content type within subprocess.py, and everything works with Py3/Django1.6 =)

长故事短，我被抓住，Django / subprocess.py正在使用环境区域设置，而不是文件/或头部Content-Type的编码。

long story short, I was caught off guard that Django/subprocess.py is using the environment locale, and not the encoding of the file / or the header Content-Type.

这篇关于Unicode问题与makemessages - 所有Django 1.6.2 Python 3.3的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！