使用多处理模块创建的进程之间的python类对象共享 [英] python Class Object Sharing between Processes created using multiprocessing module

问题描述

如何创建我的类的python共享库，可以通过辅助进程进行修改.我通过使用multiprocessing.Process模块​​创建了工作进程.我对multiprocessing.Manager模块有一些了解.谁能举例说明如何将我的班级注册到Manager中，启动Manager并创建班级的共享对象.

How do I create the python shared object of my class which can be modified by worker processes. I created the worker processes by using multiprocessing.Process module. I have some knowledge about multiprocessing.Manager module. Can any one explain with example how to register my class in to Manager, start the manager and create the shared object of my class.

推荐答案

以下是示例:

from multiprocessing import Process, Pool
from multiprocessing.managers import BaseManager


class MySharedClass(object):
    stored_value = 0
    def get(self):
        return self.stored_value

    def set(self, new_value):
        self.stored_value = new_value
        return self.stored_value


class MyManager(BaseManager):
    pass


MyManager.register('MySharedClass', MySharedClass)

def worker ( proxy_object, i):
    proxy_object.set( proxy_object.get() + i )
    print ("id %d, sum %d" %(i, proxy_object.get()))
    return proxy_object


if __name__ == '__main__':
    manager = MyManager()
    manager.start()
    shared = manager.MySharedClass()

    pool = Pool(5)
    for i in range(33):
        pool.apply(func=worker, args=(shared, i))
    pool.close()
    pool.join()
    print "result: %d" % shared.get()

id 0，总和0
ID 1，总和1
ID 2，总和3
...
id 31，总和496
id 32，总和528
结果:528

id 0, sum 0
id 1, sum 1
id 2, sum 3
...
id 31, sum 496
id 32, sum 528
result: 528

另一种变体(从未在实际项目中使用过):

Another variant (have never use it in the real project):

from multiprocessing import Process, Pool
from multiprocessing.managers import BaseManager, NamespaceProxy


class MySharedClass(object):
    def __init__(self):
        self.stored_value = 0

    def get(self):
        return self.stored_value

    def set(self, new_value):
        self.stored_value = new_value
        return self.stored_value


class MyManager(BaseManager):
    pass

class MyProxy(NamespaceProxy):
    _exposed_ = ('__getattribute__', '__setattr__', '__delattr__')# add 'get' to use get


    #def get(self):
    #    callmethod = object.__getattribute__(self, '_callmethod')
    #    return callmethod('get')

MyManager.register('MySharedClass', MySharedClass, MyProxy)

def worker ( proxy_object, i):
    proxy_object.stored_value =  proxy_object.stored_value + i
    print ("id %d, sum %d" %(i, proxy_object.stored_value))
    return proxy_object


if __name__ == '__main__':
    manager = MyManager()
    manager.start()
    shared = manager.MySharedClass()
    print shared.stored_value

    pool = Pool(5)
    for i in range(33):
        pool.apply(func=worker, args=(shared, i))
    pool.close()
    pool.join()
    print "result: %d" % shared.stored_value

这篇关于使用多处理模块创建的进程之间的python类对象共享的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！