Python:如何将列表写入文件，然后再将其拉回内存(以字符串表示的dict转换为dict)? [英] Python: How do I write a list to file and then pull it back into memory (dict represented as a string convert to dict) later?

问题描述

875228(在Python中存储简单数据)的更具体的重复方式.

More specific dupe of 875228—Simple data storing in Python.

我有一个很大的字典(6 GB)，我需要对其进行一些处理.我正在尝试几种文档聚类方法，因此我需要一次将整个内容存储在内存中.我还有其他功能可以对此数据运行，但是内容不会改变.

I have a rather large dict (6 GB) and I need to do some processing on it. I'm trying out several document clustering methods, so I need to have the whole thing in memory at once. I have other functions to run on this data, but the contents will not change.

当前，每当我想到新功能时，都必须编写它们，然后重新生成字典.我正在寻找一种将该dict写入文件的方法，以便可以将其加载到内存中，而不用重新计算所有值.

Currently, every time I think of new functions I have to write them, and then re-generate the dict. I'm looking for a way to write this dict to a file, so that I can load it into memory instead of recalculating all it's values.

过于简化的事情看起来像这样: {(((('word'，'list')，(1,2)，(1,3))，(...)):0.0，....}

to oversimplify things it looks something like: {((('word','list'),(1,2),(1,3)),(...)):0.0, ....}

我觉得python必须比我遍历一些寻找:和(试图将其解析为字典的字符串)更好的方法.

I feel that python must have a better way than me looping around through some string looking for : and ( trying to parse it into a dictionary.

推荐答案

为什么不使用 python pickle ? Python有一个很棒的序列化模块，称为pickle，非常易于使用.

Why not use python pickle? Python has a great serializing module called pickle it is very easy to use.

import cPickle
cPickle.dump(obj, open('save.p', 'wb')) 
obj = cPickle.load(open('save.p', 'rb'))

泡菜有两个缺点:

• 这是不安全的，以防止错误或错误 恶意构建的数据.绝不 解开从 不受信任或未经身份验证的来源.
• 该格式不可读.

• It's not secure against erroneous or maliciously constructed data. Never unpickle data received from an untrusted or unauthenticated source.
• The format is not human readable.

如果您使用的是python 2.6，则有一个名为 json 的内置模块.就像泡菜一样容易使用:

If you are using python 2.6 there is a builtin module called json. It is as easy as pickle to use:

import json
encoded = json.dumps(obj)
obj = json.loads(encoded)

Json格式是人类可读的格式，与python中的字典字符串表示形式非常相似.并且没有像pickle这样的任何安全性问题.但是可能比cPickle慢.

Json format is human readable and is very similar to the dictionary string representation in python. And doesn't have any security issues like pickle. But might be slower than cPickle.

这篇关于Python:如何将列表写入文件，然后再将其拉回内存(以字符串表示的dict转换为dict)?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！