ValueError:使用 ast.literal_eval 格式错误的字符串 [英] ValueError: malformed string using ast.literal_eval

问题描述

我正在循环获取 json api，这是我的循环中的内容:

response_item = requests.request('GET',url_item,params=None,verify=False)response_item = json.loads(response_item.text)response_item = ast.literal_eval(json.dumps(response_item, ensure_ascii=False).encode('utf8'))

我扫描了大约 45000 个 json 对象，我为每次迭代生成url_item"变量.每个对象都是相同的，我可以得到 7000 个对象，当我到达第 7064 个时出现以下错误:

回溯(最近一次调用最后一次):文件C:Python27	oolsapi_item.py"，第 47 行，在 <module> 中response_item = ast.literal_eval(json.dumps(response_item, ensure_ascii=False).encode('utf8'))文件C:Python27libast.py"，第 80 行，在literal_eval返回_convert(node_or_string)文件C:Python27libast.py"，第 63 行，在 _convert在 zip(node.keys, node.values))文件C:Python27libast.py"，第 62 行，在 <genexpr> 中返回 dict((_convert(k), _convert(v)) for k, v文件C:Python27libast.py"，第 63 行，在 _convert在 zip(node.keys, node.values))文件C:Python27libast.py"，第 62 行，在 <genexpr> 中返回 dict((_convert(k), _convert(v)) for k, v文件C:Python27libast.py"，第 79 行，在 _convert引发 ValueError('格式错误的字符串')值错误:格式错误的字符串

我曾经打印过第二个和第三个response_item".当然在这种情况下第三个没有显示，因为我之前有错误，这里是我在 json.load 之后打印的内容:

{u'restrictions': [], u'name': u'Sac xe0 dos de base', u'level': 0, u'rarity': u'Basic', u'vendor_value': 11, u'details': {u'no_sell_or_sort': False, u'size': 20}, u'game_types': [u'Activity', u'Wvw', u'Dungeon', u'Pve'], u'flags': [u'NoSell', u'SoulbindOnAcquire', u'SoulBindOnUse'], u'icon': u'https://render.guildwars2.com/file/80E36806385691D4C0910817EF2A6A6E73500u'type': u'Bag', u'id': 8932, u'description': u'Un sac de 20 emplacements pour les personnages dxe9butants.'}

在这之前我得到的每一项都具有相同的类型、相同的格式，除了第 7064 项之外我没有任何错误！

感谢您的帮助！

解决方案

你应该不要对 JSON 数据使用 ast.literal_eval().JSON 和 Python 文字可能看起来很像，但实际上并非如此.

在这种情况下，您的数据包含一个布尔标志，在 JSON 中设置为 false.正确的 Python 布尔值使用标题大小写，所以 False:

<预><代码>>>>导入 json, ast>>>s = '{"no_sell_or_sort": false, "size": 20}'>>>json.loads(s){u'no_sell_or_sort': False, u'size': 20}>>>ast.literal_eval(s)回溯(最近一次调用最后一次):文件<stdin>"，第 1 行，在 <module> 中文件/Users/mj/Development/Library/buildout.python/parts/opt/lib/python2.7/ast.py"，第80行，在literal_eval中返回_convert(node_or_string)文件/Users/mj/Development/Library/buildout.python/parts/opt/lib/python2.7/ast.py"，第 63 行，在 _convert在 zip(node.keys, node.values))文件/Users/mj/Development/Library/buildout.python/parts/opt/lib/python2.7/ast.py"，第62行，在<genexpr>返回 dict((_convert(k), _convert(v)) for k, v文件/Users/mj/Development/Library/buildout.python/parts/opt/lib/python2.7/ast.py"，第 79 行，在 _convert引发 ValueError('格式错误的字符串')值错误:格式错误的字符串

其他差异包括使用 null 而不是 None，以及 Unicode 转义序列在 Python 2 中看起来像一个普通(字节)字符串, 在转义非 BMP 代码点时使用 UTF-16 代理.

使用 json.loads() 加载数据，而不是 ast.literal_eval().它不仅可以很好地处理正确的 JSON，而且更快.

在您的情况下，您似乎正在使用 json.dumps() 然后尝试使用 ast.literal_eval() 再次加载数据.这一步不需要，您已经有一个 Python 对象.

换句话说，该行:

response_item = ast.literal_eval(json.dumps(response_item, ensure_ascii=False).encode('utf8'))

充其量是多余的，最坏的情况是非常非常错误.将 response_item 重新编码为 JSON 字符串不会产生可以解释为 Python 文字的内容.

I'm doing a loop to get json api, here is what I have in my loop:

response_item = requests.request('GET',url_item,params=None,verify=False)
response_item = json.loads(response_item.text)
response_item = ast.literal_eval(json.dumps(response_item, ensure_ascii=False).encode('utf8'))

I scan around 45000 json objects, I generate "url_item" variable for each iteration. Each object is the same, I can get something like 7000 object and I have the following error when I reach the 7064th:

Traceback (most recent call last):
  File "C:Python27	oolsapi_item.py", line 47, in <module>
    response_item = ast.literal_eval(json.dumps(response_item, ensure_ascii=False).encode('utf8'))
  File "C:Python27libast.py", line 80, in literal_eval
    return _convert(node_or_string)
  File "C:Python27libast.py", line 63, in _convert
    in zip(node.keys, node.values))
  File "C:Python27libast.py", line 62, in <genexpr>
    return dict((_convert(k), _convert(v)) for k, v
  File "C:Python27libast.py", line 63, in _convert
    in zip(node.keys, node.values))
  File "C:Python27libast.py", line 62, in <genexpr>
    return dict((_convert(k), _convert(v)) for k, v
  File "C:Python27libast.py", line 79, in _convert
    raise ValueError('malformed string')
ValueError: malformed string

I used to print the second and third "response_item". Of course in this case the third one isn't displayed since I have the error just before, here what I have for the print after the json.load:

{u'restrictions': [], u'name': u'Sac xe0 dos de base', u'level': 0, u'rarity': u'Basic', u'vendor_value': 11, u'details': {u'no_sell_or_sort': False, u'size': 20}, u'game_types': [u'Activity', u'Wvw', u'Dungeon', u'Pve'], u'flags': [u'NoSell', u'SoulbindOnAcquire', u'SoulBindOnUse'], u'icon': u'https://render.guildwars2.com/file/80E36806385691D4C0910817EF2A6C2006AEE353/61755.png', u'type': u'Bag', u'id': 8932, u'description': u'Un sac de 20 emplacements pour les personnages dxe9butants.'}

Every item I get before this one has the same type, same format, and I don't have any error except for the 7064th !

Thank you for your help!

解决方案

You should not use ast.literal_eval() on JSON data. JSON and Python literals may look like the same thing, but they are very much not.

In this case, your data contains a boolean flag, set to false in JSON. A proper Python boolean uses title-case, so False:

>>> import json, ast
>>> s = '{"no_sell_or_sort": false, "size": 20}'
>>> json.loads(s)
{u'no_sell_or_sort': False, u'size': 20}
>>> ast.literal_eval(s)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/mj/Development/Library/buildout.python/parts/opt/lib/python2.7/ast.py", line 80, in literal_eval
    return _convert(node_or_string)
  File "/Users/mj/Development/Library/buildout.python/parts/opt/lib/python2.7/ast.py", line 63, in _convert
    in zip(node.keys, node.values))
  File "/Users/mj/Development/Library/buildout.python/parts/opt/lib/python2.7/ast.py", line 62, in <genexpr>
    return dict((_convert(k), _convert(v)) for k, v
  File "/Users/mj/Development/Library/buildout.python/parts/opt/lib/python2.7/ast.py", line 79, in _convert
    raise ValueError('malformed string')
ValueError: malformed string

Other differences include using null instead of None, and Unicode escape sequences in what to Python 2 looks like a plain (bytes) string, using UTF-16 surrogates when escaping non-BMP codepoints.

Load your data with json.loads(), not ast.literal_eval(). Not only will it handle proper JSON just fine, it is also faster.

In your case, it appears you are using json.dumps() then try to load the data again with ast.literal_eval(). There is no need for that step, you already had a Python object.

In other words, the line:

response_item = ast.literal_eval(json.dumps(response_item, ensure_ascii=False).encode('utf8'))

is redundant at best, and very, very wrong, at worst. Re-encoding response_item to a JSON string does not produce something that can be interpreted as a Python literal.

这篇关于ValueError:使用 ast.literal_eval 格式错误的字符串的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！