urllib2 opener 提供错误的字符集 [英] urllib2 opener providing wrong charset

问题描述

当我打开网址阅读时，我无法识别它.但是当我检查内容标题时，它说它被编码为 utf-8.所以我试图将它转换为 unicode，它抱怨 UnicodeDecodeError: 'ascii' codec can't decode byte 0x8b in position 1: ordinal not in range(128) using unicode().

When I open the url and read it, I can't recognize it. But when I check the content header it says it is encoded as utf-8. So I tried to convert it to unicode and it complained UnicodeDecodeError: 'ascii' codec can't decode byte 0x8b in position 1: ordinal not in range(128) using unicode().

.encode("utf-8") 产生UnicodeDecodeError:ascii"编解码器无法解码位置 1 中的字节 0x8b:序号不在范围内 (128)

.encode("utf-8") produces UnicodeDecodeError: 'ascii' codec can't decode byte 0x8b in position 1: ordinal not in range(128)

.decode("utf-8") 产生UnicodeDecodeError: 'utf8' 编解码器无法解码位置 1 的字节 0x8b:起始字节无效.

.decode("utf-8") produced UnicodeDecodeError: 'utf8' codec can't decode byte 0x8b in position 1: invalid start byte.

我已经尝试了我能想到的所有方法(我不太擅长编码)

I have tried everything I can come up with(I'm not that good at encodings)

如果我能让它工作，我会很高兴.谢谢.

I would be happy if I could get this to work. Thanks.

推荐答案

这是一个常见的错误.服务器发送 gzip 压缩流.

This is a common mistake. The server sends gzipped stream.

你应该先打开它:

response = opener.open(self.__url, data)
if response.info().get('Content-Encoding') == 'gzip':
    buf = StringIO.StringIO( response.read())
    gzip_f = gzip.GzipFile(fileobj=buf)
    content = gzip_f.read()
else:
    content = response.read()

这篇关于urllib2 opener 提供错误的字符集的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！