什么是更快的-将腌制的字典对象或JSON文件加载到字典? [英] What is faster - Loading a pickled dictionary object or Loading a JSON file - to a dictionary?

问题描述

什么更快?

(A)使用pickle.load()

或

(B)使用simplejson.load()

假设:如果情况A，则已腌制的目标文件已经存在， 并且在情况B中该JSON文件已经存在.

Assuming: The pickled object file exists already in case A, and that the JSON file exists already in case B.

推荐答案

速度实际上取决于数据，数据的内容和大小.

The speed actually depends on the data, it's content and size.

但是，无论如何，让我们以json数据为例，看看有什么更快的方法(Ubuntu 12.04，python 2.7.3):

But, anyway, let's take an example json data and see what is faster (Ubuntu 12.04, python 2.7.3) :

• 腌制
• cPickle
• json
• simplejson
• ujson
• yajl

• pickle
• cPickle
• json
• simplejson
• ujson
• yajl

将这个json结构转储到test.json和test.pickle文件中:

Giving this json structure dumped into test.json and test.pickle files:

{
    "glossary": {
        "title": "example glossary",
        "GlossDiv": {
            "title": "S",
            "GlossList": {
                "GlossEntry": {
                    "ID": "SGML",
                    "SortAs": "SGML",
                    "GlossTerm": "Standard Generalized Markup Language",
                    "Acronym": "SGML",
                    "Abbrev": "ISO 8879:1986",
                    "GlossDef": {
                        "para": "A meta-markup language, used to create markup languages such as DocBook.",
                        "GlossSeeAlso": ["GML", "XML"]
                    },
                    "GlossSee": "markup"
                }
            }
        }
    }
}

测试脚本:

import timeit

import pickle
import cPickle

import json
import simplejson
import ujson
import yajl


def load_pickle(f):
    return pickle.load(f)


def load_cpickle(f):
    return cPickle.load(f)


def load_json(f):
    return json.load(f)


def load_simplejson(f):
    return simplejson.load(f)


def load_ujson(f):
    return ujson.load(f)


def load_yajl(f):
    return yajl.load(f)


print "pickle:"
print timeit.Timer('load_pickle(open("test.pickle"))', 'from __main__ import load_pickle').timeit()

print "cpickle:"
print timeit.Timer('load_cpickle(open("test.pickle"))', 'from __main__ import load_cpickle').timeit()

print "json:"
print timeit.Timer('load_json(open("test.json"))', 'from __main__ import load_json').timeit()

print "simplejson:"
print timeit.Timer('load_simplejson(open("test.json"))', 'from __main__ import load_simplejson').timeit()

print "ujson:"
print timeit.Timer('load_ujson(open("test.json"))', 'from __main__ import load_ujson').timeit()

print "yajl:"
print timeit.Timer('load_yajl(open("test.json"))', 'from __main__ import load_yajl').timeit()

输出:

pickle:
107.936687946

cpickle:
28.4231381416

json:
31.6450419426

simplejson:
20.5853149891

ujson:
16.9352178574

yajl:
18.9763481617

如您所见，通过pickle进行的酸洗根本不是那么快-如果选择酸洗/酸洗"选项，cPickle是明确的选择.在这些特定数据的这些json解析器中，ujson看起来很有希望.

As you can see, unpickling via pickle is not that fast at all - cPickle is definetely the way to go if you choose pickling/unpickling option. ujson looks promising among these json parsers on this particular data.

此外，json和simplejson库在 pypy 上的加载速度要快得多(请参见

Also, json and simplejson libraries load much faster on pypy (see Python JSON Performance).

另请参阅:

• Python JSON解码性能
• 是腌制还是json?
• Pickle与JSON —哪个更快?

• Python JSON decoding performance
• Pickle or json?
• Pickle vs JSON — Which is Faster?

请务必注意，结果可能会因您的特定系统，其他类型和数据大小而有所不同.

It's important to note that the results may differ on your particular system, on other type and size of data.

这篇关于什么是更快的-将腌制的字典对象或JSON文件加载到字典?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！