Python 3,从/向gzip文件读取/写入压缩的json对象 [英] Python 3, read/write compressed json objects from/to gzip file
本文介绍了Python 3,从/向gzip文件读取/写入压缩的json对象的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
对于Python3,我关注了 @Martijn彼得的代码与此:
For Python3, I followed @Martijn Pieters's code with this:
import gzip
import json
# writing
with gzip.GzipFile(jsonfilename, 'w') as fout:
for i in range(N):
uid = "whatever%i" % i
dv = [1, 2, 3]
data = json.dumps({
'what': uid,
'where': dv})
fout.write(data + '\n')
但这会导致错误:
Traceback (most recent call last):
...
File "C:\Users\Think\my_json.py", line 118, in write_json
fout.write(data + '\n')
File "C:\Users\Think\Anaconda3\lib\gzip.py", line 258, in write
data = memoryview(data)
TypeError: memoryview: a bytes-like object is required, not 'str'
有什么想法吗?
推荐答案
这里有四个转换步骤.
- Python数据结构(嵌套字典,列表,字符串,数字,布尔值)
- 包含该数据结构("JSON")的序列化表示形式的Python字符串
- 包含该字符串表示形式的字节列表("UTF-8")
- 包含先前字节列表("gzip")表示形式的字节列表
因此,让我们一步一步地进行这些步骤.
So let's take these steps one by one.
import gzip
import json
data = []
for i in range(N):
uid = "whatever%i" % i
dv = [1, 2, 3]
data.append({
'what': uid,
'where': dv
}) # 1. data
json_str = json.dumps(data) + "\n" # 2. string (i.e. JSON)
json_bytes = json_str.encode('utf-8') # 3. bytes (i.e. UTF-8)
with gzip.GzipFile(jsonfilename, 'w') as fout: # 4. gzip
fout.write(json_bytes)
请注意,在这里添加"\n"
是完全多余的.它不会破坏任何东西,但是除此之外,它毫无用处.
Note that adding "\n"
is completely superfluous here. It does not break anything, but beyond that it has no use.
阅读工作完全相反:
with gzip.GzipFile(jsonfilename, 'r') as fin: # 4. gzip
json_bytes = fin.read() # 3. bytes (i.e. UTF-8)
json_str = json_bytes.decode('utf-8') # 2. string (i.e. JSON)
data = json.loads(json_str) # 1. data
print(data)
当然可以将步骤组合在一起
Of course the steps can be combined:
with gzip.GzipFile(jsonfilename, 'w') as fout:
fout.write(json.dumps(data).encode('utf-8'))
和
with gzip.GzipFile(jsonfilename, 'r') as fin:
data = json.loads(fin.read().decode('utf-8'))
这篇关于Python 3,从/向gzip文件读取/写入压缩的json对象的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!
查看全文