腌制和复制持久对象的类? [英] Class for pickle- and copy-persistent object?

问题描述

我正在尝试为一个只读对象编写一个类，该类将不会真正使用copy模块进行复制，并且当将其腌制在进程之间进行传输时，每个进程将维护不超过一个副本其中，无论将它作为新"对象传递多少次.已经有这样的东西了吗?

I'm trying to write a class for a read-only object which will not be really copied with the copy module, and when it will be pickled to be transferred between processes each process will maintain no more than one copy of it, no matter how many times it will be passed around as a "new" object. Is there already something like that?

推荐答案

我尝试实现这一点. @Alex Martelli和其他任何人，请给我评论/改进.我认为这最终将在GitHub上结束.

I made an attempt to implement this. @Alex Martelli and anyone else, please give me comments/improvements. I think this will eventually end up on GitHub.

"""
todo: need to lock library to avoid thread trouble?

todo: need to raise an exception if we're getting pickled with
an old protocol?

todo: make it polite to other classes that use __new__. Therefore, should
probably work not only when there is only one item in the *args passed to new.

"""

import uuid
import weakref

library = weakref.WeakValueDictionary()

class UuidToken(object):
    def __init__(self, uuid):
        self.uuid = uuid


class PersistentReadOnlyObject(object):
    def __new__(cls, *args, **kwargs):
        if len(args)==1 and len(kwargs)==0 and isinstance(args[0], UuidToken):
            received_uuid = args[0].uuid
        else:
            received_uuid = None

        if received_uuid:
            # This section is for when we are called at unpickling time
            thing = library.pop(received_uuid, None)
            if thing:
                thing._PersistentReadOnlyObject__skip_setstate = True
                return thing
            else: # This object does not exist in our library yet; Let's add it
                new_args = args[1:]
                thing = super(PersistentReadOnlyObject, cls).__new__(cls,
                                                                     *new_args,
                                                                     **kwargs)
                thing._PersistentReadOnlyObject__uuid = received_uuid
                library[received_uuid] = thing
                return thing

        else:
            # This section is for when we are called at normal creation time
            thing = super(PersistentReadOnlyObject, cls).__new__(cls, *args,
                                                                 **kwargs)
            new_uuid = uuid.uuid4()
            thing._PersistentReadOnlyObject__uuid = new_uuid
            library[new_uuid] = thing
            return thing

    def __getstate__(self):
        my_dict = dict(self.__dict__)
        del my_dict["_PersistentReadOnlyObject__uuid"]
        return my_dict

    def __getnewargs__(self):
        return (UuidToken(self._PersistentReadOnlyObject__uuid),)

    def __setstate__(self, state):
        if self.__dict__.pop("_PersistentReadOnlyObject__skip_setstate", None):
            return
        else:
            self.__dict__.update(state)

    def __deepcopy__(self, memo):
        return self

    def __copy__(self):
        return self

# --------------------------------------------------------------
"""
From here on it's just testing stuff; will be moved to another file.
"""


def play_around(queue, thing):
    import copy
    queue.put((thing, copy.deepcopy(thing),))

class Booboo(PersistentReadOnlyObject):
    def __init__(self):
        self.number = random.random()

if __name__ == "__main__":

    import multiprocessing
    import random
    import copy

    def same(a, b):
        return (a is b) and (a == b) and (id(a) == id(b)) and \
               (a.number == b.number)

    a = Booboo()
    b = copy.copy(a)
    c = copy.deepcopy(a)
    assert same(a, b) and same(b, c)

    my_queue = multiprocessing.Queue()
    process = multiprocessing.Process(target = play_around,
                                      args=(my_queue, a,))
    process.start()
    process.join()
    things = my_queue.get()
    for thing in things:
        assert same(thing, a) and same(thing, b) and same(thing, c)
    print("all cool!")

这篇关于腌制和复制持久对象的类?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！