跨进程共享对象状态? [英] Share object state across processes?

问题描述

在下面的代码中，如何使Starter对象能够读取gen.vals? 似乎创建了一个不同的对象，其状态得到了更新，但是Starter从来都不知道.另外，对于self.vals是字典还是任何其他类型的对象，该解决方案将如何应用?

In the code below, how do I make the Starter object be able to read gen.vals? It seems like a different object gets created, whose state gets updated, but Starter never knows about it. Also, how would the solution apply for self.vals being a dictionary, or any other kind of object?

import multiprocessing
import time

class Generator(multiprocessing.Process):
    def __init__(self):
        self.vals = []
        super(Generator, self).__init__()

    def run(self):
        i = 0
        while True:
            time.sleep(1)
            self.vals.append(i)
            print 'In Generator ', self.vals # prints growing list
            i += 1

class Starter():
    def do_stuff(self):
        gen = Generator()
        gen.start()
        while True:
            print 'In Starter ', gen.vals # prints empty list
            time.sleep(1)

if __name__ == '__main__':
    starter = Starter()
    starter.do_stuff()

输出:

In Starter  []
In Starter  []
In Generator  [0]
In Starter  []
In Generator  [0, 1]
In Starter  []
In Generator  [0, 1, 2]
In Starter  []
In Generator  [0, 1, 2, 3]
In Starter  []
In Generator  [0, 1, 2, 3, 4]
In Starter  []
In Generator  [0, 1, 2, 3, 4, 5]
In Starter  []
In Generator  [0, 1, 2, 3, 4, 5, 6]
In Starter  []
In Generator  [0, 1, 2, 3, 4, 5, 6, 7]

推荐答案

启动流程时，它实际上是在整个单独的上下文中执行的(这是

When you start a process it essentially executes in a whole separate context (here's a brief explanation on what's going on) so there is no shared memory to speak of, therefore whatever your run() method does doesn't really reflect in your main process - Python spawns/forks a whole new process out of it, instantiates your Generator there and calls its run() method and any changes to the state of that other instance in a different process stay there.

如果要传递数据，则需要使用一些具有多处理能力的结构，这些结构实际上将对不同进程之间的数据进行序列化/反序列化，并前后传递更改.例如:

If you want to pass data around, you need to use some multiprocessing aware structures that will essentially serialize/deserialize data between different processes and communicate the changes back and forward. For example:

import multiprocessing
import time

class Generator(multiprocessing.Process):
    def __init__(self):
        self._vals = []  # keeps the internal state
        self.vals = multiprocessing.Queue()  # a queue for the exchange
        super(Generator, self).__init__()

    def run(self):
        i = 0
        while True:
            time.sleep(1)
            self._vals.append(i)  # update the internal state
            print('In Generator ', self._vals) # prints growing list
            self.vals.put(self._vals)  # add it to the queue
            i += 1

class Starter():
    def do_stuff(self):
        gen = Generator()
        gen.start()
        while True:
            print('In Starter ', gen.vals.get()) # print what's in the queue
            time.sleep(1)

if __name__ == '__main__':
    starter = Starter()
    starter.do_stuff()

将打印出:

In Generator  [0]
In Starter  [0]
In Generator  [0, 1]
In Starter  [0, 1]
In Generator  [0, 1, 2]
In Starter  [0, 1, 2]
In Generator  [0, 1, 2, 3]
In Starter  [0, 1, 2, 3]
etc.

如果您要进行更复杂/半并行的数据修改或处理更多的结构化数据，请检查 Redis 过程数据交换.或者，如果您希望自己进行微观管理，ØMQ总是一个不错的选择.

If you want to do more complex/semi-concurrent data modifications or deal with more structured data, check the structures supported by multiprocessing.Manager. Of course, for very complex stuff I'd always recommend using an in-memory database like Redis as a means of inter-process data exchange. Or, if you prefer to do micro-management yourself, ØMQ is always a good option.

这篇关于跨进程共享对象状态?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！