Python 3，从/向gzip文件读取/写入压缩的json对象 [英] Python 3, read/write compressed json objects from/to gzip file

问题描述

对于Python3，我关注了 @Martijn彼得的代码与此:

For Python3, I followed @Martijn Pieters's code with this:

import gzip
import json

# writing
with gzip.GzipFile(jsonfilename, 'w') as fout:
    for i in range(N):
        uid = "whatever%i" % i
        dv = [1, 2, 3]
        data = json.dumps({
            'what': uid,
            'where': dv})

        fout.write(data + '\n')

但这会导致错误:

Traceback (most recent call last):
    ...
  File "C:\Users\Think\my_json.py", line 118, in write_json
    fout.write(data + '\n')
  File "C:\Users\Think\Anaconda3\lib\gzip.py", line 258, in write
    data = memoryview(data)
TypeError: memoryview: a bytes-like object is required, not 'str'

有什么想法吗?

推荐答案

这里有四个转换步骤.

1. Python数据结构(嵌套字典，列表，字符串，数字，布尔值)
2. 包含该数据结构("JSON")的序列化表示形式的Python字符串
3. 包含该字符串表示形式的字节列表("UTF-8")
4. 包含先前字节列表("gzip")表示形式的字节列表

因此，让我们一步一步地进行这些步骤.

So let's take these steps one by one.

import gzip
import json

data = []
for i in range(N):
    uid = "whatever%i" % i
    dv = [1, 2, 3]
    data.append({
        'what': uid,
        'where': dv
    })                                           # 1. data

json_str = json.dumps(data) + "\n"               # 2. string (i.e. JSON)
json_bytes = json_str.encode('utf-8')            # 3. bytes (i.e. UTF-8)

with gzip.GzipFile(jsonfilename, 'w') as fout:   # 4. gzip
    fout.write(json_bytes)                       

请注意，在这里添加"\n"是完全多余的.它不会破坏任何东西，但是除此之外，它毫无用处.

Note that adding "\n" is completely superfluous here. It does not break anything, but beyond that it has no use.

阅读工作完全相反:

with gzip.GzipFile(jsonfilename, 'r') as fin:    # 4. gzip
    json_bytes = fin.read()                      # 3. bytes (i.e. UTF-8)

json_str = json_bytes.decode('utf-8')            # 2. string (i.e. JSON)
data = json.loads(json_str)                      # 1. data

print(data)

当然可以将步骤组合在一起

Of course the steps can be combined:

with gzip.GzipFile(jsonfilename, 'w') as fout:
    fout.write(json.dumps(data).encode('utf-8'))                       

和

with gzip.GzipFile(jsonfilename, 'r') as fin:
    data = json.loads(fin.read().decode('utf-8'))

这篇关于Python 3，从/向gzip文件读取/写入压缩的json对象的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！