在Python中将非标准字符解码为UTF 8 [英] Decoding non standard characters to UTF 8 in Python

问题描述

我有一个程序，该程序通过Django中的一个Webhook接收字节编码的文本(用Python编写).我已经从字节解码-> utf-8正常字母工作，但是在发送撇号(')时它会中断.我编写此代码是为了解码文本:

I have a program that takes in byte-encoded text via a webhook in Django (written in Python). I have decoding from byte -> utf-8 working for normal letters, but it breaks when an apostrophe ( ' ) is sent in. I have this written to decode the text:

encoded = request.body
decoded = parse_qs(encoded)
body = decoded[b'body'][0].decode("utf-8")

这是错误:

UnicodeEncodeError: 'ascii' codec can't encode character '\u2019' in position 5: ordinal not in range(128)

我希望它能够成功解码撇号.我还担心如果发送表情符号可能会中断，所以我希望能够转义表情符号和∫这样的随机字符，但仍保留消息中的真实单词.

I'd like for it to successfully decode apostrophes. I'm also concerned it might break if an emoji is sent in, so I'd like to be able to escape emoji and random chars like ∫, but still preserve the real words in the message.

推荐答案

parse_qs将与解码的utf字符串一起使用，但在非ascii字节上造成阻塞.例如:

parse_qs will work with decoded utf strings but chokes on non-ascii bytes. For example:

失败:

a = b'restaurant_type=caf\xc3\xa9'
urllib.parse.parse_qs(a)
# > UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3...etc

但这行得通:

a = b'restaurant_type=caf\xc3\xa9'
urllib.parse.parse_qs(a.decode())
# > {'restaurant_type': ['café']}

您在问什么吗?

这篇关于在Python中将非标准字符解码为UTF 8的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！