copy.deepcopy与泡菜 [英] copy.deepcopy vs pickle

问题描述

我有一个小部件的树状结构，例如集合包含模型，模型包含小部件.我要复制整个集合，与将对象刺穿和去刺穿"相比，copy.deepcopy更快，但是cPickle因为用C语言编写要快得多，所以

I have a tree structure of widgets e.g. collection contains models and model contains widgets. I want to copy whole collection, copy.deepcopy is faster in comparison to 'pickle and de-pickle'ing the object but cPickle as being written in C is much faster, so

1. 为什么我(我们)不总是使用cPickle而不是deepcopy?
2. 还有其他复印选择吗?因为pickle比Deepcopy慢，但是cPickle更快，所以可能是Deepcopy的C实现会成为赢家

示例测试代码:

import copy
import pickle
import cPickle

class A(object): pass

d = {}
for i in range(1000):
    d[i] = A()

def copy1():
    return copy.deepcopy(d)

def copy2():
    return pickle.loads(pickle.dumps(d, -1))

def copy3():
    return cPickle.loads(cPickle.dumps(d, -1))

时间:

>python -m timeit -s "import c" "c.copy1()"
10 loops, best of 3: 46.3 msec per loop

>python -m timeit -s "import c" "c.copy2()"
10 loops, best of 3: 93.3 msec per loop

>python -m timeit -s "import c" "c.copy3()"
100 loops, best of 3: 17.1 msec per loop

推荐答案

问题是，pickle + unpickle可以更快(在C语言实现中)，因为它比deepcopy的通用性更强:许多对象可以深层复制但不腌制.例如，假设您的班级A更改为...:

Problem is, pickle+unpickle can be faster (in the C implementation) because it's less general than deepcopy: many objects can be deepcopied but not pickled. Suppose for example that your class A were changed to...:

class A(object):
  class B(object): pass
  def __init__(self): self.b = self.B()

现在，copy1仍然可以正常工作(A的复杂性使其速度降低，但绝对不能阻止它)； copy2和copy3中断，堆栈跟踪的末尾显示...:

now, copy1 still works fine (A's complexity slows it downs but absolutely doesn't stop it); copy2 and copy3 break, the end of the stack trace says...:

  File "./c.py", line 20, in copy3
    return cPickle.loads(cPickle.dumps(d, -1))
PicklingError: Can't pickle <class 'c.B'>: attribute lookup c.B failed

也就是说，酸洗总是假定类和函数是其模块中的顶级实体，因此，按名称"对它们进行酸洗-深度复制绝对没有这样的假设.

I.e., pickling always assumes that classes and functions are top-level entities in their modules, and so pickles them "by name" -- deepcopying makes absolutely no such assumptions.

因此，如果您遇到某种程度的深度复制"的速度至关重要的情况，那么每一毫秒都很重要，并且您想利用您知道要应用于复制对象的特殊限制，例如那些使酸洗适用的方法，或偏爱序列化和其他快捷方式的其他形式的方法，请务必继续进行-但是，如果您这样做，则必须意识到您将永远限制这些系统的生存，并记录该设计决策非常清楚，明确地为将来的维护者造福.

So if you have a situation where speed of "somewhat deep-copying" is absolutely crucial, every millisecond matters, AND you want to take advantage of special limitations that you KNOW apply to the objects you're duplicating, such as those that make pickling applicable, or ones favoring other forms yet of serializations and other shortcuts, by all means go ahead - but if you do you MUST be aware that you're constraining your system to live by those limitations forevermore, and document that design decision very clearly and explicitly for the benefit of future maintainers.

对于普通情况，需要通用性，请使用deepcopy！-)

For the NORMAL case, where you want generality, use deepcopy!-)

这篇关于copy.deepcopy与泡菜的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！