python - 正则表达式和unicode的问题 [英] python - problems with regular expression and unicode

问题描述

我在使用 python 时遇到问题.我试着用一个例子来解释我的问题.

我有这个字符串:

<预><代码>>>>string = 'ÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿÀÁÂÃ'>>>打印字符串ÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìííîðñòóôõö÷øùúûüýþÿÀÁÂÃ

例如，我想用"替换与 Ñ,Ã,ï 不同的字符

我已经尝试过:

<预><代码>>>>rePat = re.compile('[^ÑÃï]',re.UNICODE)>>>打印 rePat.sub("",string)Ñ

我得到了这个 .我认为这是因为python中的这种类型的字符由向量中的两个位置表示:例如\xc3\x91 = Ñ.为此，当我进行 regolar 表达式时，所有的 \xc3 都不会被替换.我怎么能做这种类型的子??????

谢谢佛朗哥

解决方案

您需要确保您的字符串是 unicode 字符串，而不是纯字符串(纯字符串类似于字节数组).

示例:

<预><代码>>>>string = 'ÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿÀÁÂÃ'>>>类型(字符串)<输入'str'># 改为这样做:#(注意 ' 前面的 u，这将字符序列标记为 unicode 文字)>>>字符串 = u'\xd0\xd1\xd2\xd3\xd4\xd5\xd6\xd7\xd8\xd9\xda\xdb\xdc\xdd\xde\xdf\xe0\xe1\xe2\xe3\xe4\xe5\xe6\xe7\xe8\xe9\xea\xeb\xec\xed\xee\xef\xf0\xf1\xf2\xf3\xf4\xf5\xf6\xf7\xf8\xf9\xfa\xfb\xfc\xfd\xfe\xff\xc0\xc1\xc2\xc3'# 或者:>>>string = 'ÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿÀÁÃ'.decode('utf-8')# ...但请注意，后者仅在终端(或源文件)具有 utf-8 编码时才有效# ... 最佳实践是在 unicode 文字中使用 \xNN 形式，如第一个示例所示>>>类型(字符串)<输入'unicode'>>>>打印字符串ÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìííîðñòóôõö÷øùúûüýþÿÀÁÂÃ>>>rePat = re.compile(u'[^\xc3\x91\xc3\x83\xc3\xaf]',re.UNICODE)>>>打印 rePat.sub("", string)一种

<小时>

读取文件时，string = open('filename.txt').read()读取一个字节序列.

要获取 unicode 内容，请执行:string = unicode(open('filename.txt').read(), 'encoding').或者:string = open('filename.txt').read().decode('encoding').

codecs 模块可以解码 unicode 流(例如文件)——飞.

在谷歌上搜索 python unicode.Python unicode 处理一开始可能有点难以掌握，需要仔细阅读.

我遵守这条规则:软件应该只在内部使用 Unicode 字符串，在输出时转换为特定的编码."(来自 http://www.amk.ca/python/howto/unicode)

我还推荐:http://www.joelonsoftware.com/articles/Unicode.html

Hi I have a problem in python. I try to explain my problem with an example.

I have this string:

>>> string = 'ÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿÀÁÂÃ'
>>> print string
ÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿÀÁÂÃ

and i want, for example, replace charachters different from Ñ,Ã,ï with ""

i have tried:

>>> rePat = re.compile('[^ÑÃï]',re.UNICODE)
>>> print rePat.sub("",string)
�Ñ�����������������������������ï�������������������Ã

I obtained this �. I think that it's happen because this type of characters in python are represented by two position in the vector: for example \xc3\x91 = Ñ. For this, when i make the regolar expression, all the \xc3 are not substitued. How I can do this type of sub?????

Thanks Franco

解决方案

You need to make sure that your strings are unicode strings, not plain strings (plain strings are like byte arrays).

Example:

>>> string = 'ÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿÀÁÂÃ'
>>> type(string)
<type 'str'>

# do this instead:
# (note the u in front of the ', this marks the character sequence as a unicode literal)
>>> string = u'\xd0\xd1\xd2\xd3\xd4\xd5\xd6\xd7\xd8\xd9\xda\xdb\xdc\xdd\xde\xdf\xe0\xe1\xe2\xe3\xe4\xe5\xe6\xe7\xe8\xe9\xea\xeb\xec\xed\xee\xef\xf0\xf1\xf2\xf3\xf4\xf5\xf6\xf7\xf8\xf9\xfa\xfb\xfc\xfd\xfe\xff\xc0\xc1\xc2\xc3'
# or:
>>> string = 'ÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿÀÁÂÃ'.decode('utf-8')
# ... but be aware that the latter will only work if the terminal (or source file) has utf-8 encoding
# ... it is a best practice to use the \xNN form in unicode literals, as in the first example

>>> type(string)
<type 'unicode'>
>>> print string
ÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿÀÁÂÃ

>>> rePat = re.compile(u'[^\xc3\x91\xc3\x83\xc3\xaf]',re.UNICODE)
>>> print rePat.sub("", string)
Ã

---

When reading from a file, string = open('filename.txt').read() reads a byte sequence.

To get the unicode content, do: string = unicode(open('filename.txt').read(), 'encoding'). Or: string = open('filename.txt').read().decode('encoding').

The codecs module can decode unicode streams (such as files) on-the-fly.

Do a google search for python unicode. Python unicode handling can be a bit hard to grasp at first, it pays to read up on it.

I live by this rule: "Software should only work with Unicode strings internally, converting to a particular encoding on output." (from http://www.amk.ca/python/howto/unicode)

I also recommend: http://www.joelonsoftware.com/articles/Unicode.html

这篇关于python - 正则表达式和unicode的问题的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！