如何使用 multiprocessing.Manager()? [英] How to use a multiprocessing.Manager()?

问题描述

我担心 Python 中的 multiprocessing.Manager().例子如下:

I have a concern about multiprocessing.Manager() in python. Here is the example:

import multiprocessing

def f(ns):
    ns.x *=10
    ns.y *= 10

if __name__ == '__main__':
    manager = multiprocessing.Manager()
    ns = manager.Namespace()
    ns.x = 1
    ns.y = 2

    print 'before', ns
    p = multiprocessing.Process(target=f, args=(ns,))
    p.start()
    p.join()
    print 'after', ns

输出为:

before Namespace(x=1, y=2)
after Namespace(x=10, y=20)

直到现在，它都按我的预期工作，然后我像这样修改了代码:

Until now, it worked as I expected, then I modified the code like this:

import multiprocessing

def f(ns):
    ns.x.append(10)
    ns.y.append(10)

if __name__ == '__main__':
    manager = multiprocessing.Manager()
    ns = manager.Namespace()
    ns.x = []
    ns.y = []

    print 'before', ns
    p = multiprocessing.Process(target=f, args=(ns,))
    p.start()
    p.join()
    print 'after', ns

现在输出是:

before Namespace(x=[], y=[])
after Namespace(x=[], y=[])

令我困惑的是为什么列表没有按照我的预期进行更改.谁能帮我弄清楚发生了什么?

It confuses me why the lists were not changed as I expected. Can anyone help me to figure out what happened?

推荐答案

管理器代理对象无法传播对容器内(非托管)可变对象所做的更改.因此，换句话说，如果您有一个 manager.list() 对象，则对托管列表本身的任何更改都会传播到所有其他进程.但是，如果您有一个普通的 Python 列表inside，那么对内部列表的任何更改都不会传播，因为管理器无法检测到更改.

Manager proxy objects are unable to propagate changes made to (unmanaged) mutable objects inside a container. So in other words, if you have a manager.list() object, any changes to the managed list itself are propagated to all the other processes. But if you have a normal Python list inside that list, any changes to the inner list are not propagated, because the manager has no way of detecting the change.

为了传播更改，您还必须对嵌套列表使用 manager.list() 对象(需要 Python 3.6 或更高版本)，或者您需要直接修改 manager.list() 对象(请参阅注释 在 Python 3.5 中的 manager.list或以上).

In order to propagate the changes, you have to use manager.list() objects for the nested lists too (requires Python 3.6 or newer), or you need to modify the manager.list() object directly (see the note on manager.list in Python 3.5 or older).

例如，考虑以下代码及其输出:

For example, consider the following code and its output:

import multiprocessing
import time

def f(ns, ls, di):
    ns.x += 1
    ns.y[0] += 1
    ns_z = ns.z
    ns_z[0] += 1
    ns.z = ns_z

    ls[0] += 1
    ls[1][0] += 1 # unmanaged, not assigned back
    ls_2 = ls[2]  # unmanaged...
    ls_2[0] += 1
    ls[2] = ls_2  # ... but assigned back
    ls[3][0] += 1 # managed, direct manipulation

    di[0] += 1
    di[1][0] += 1 # unmanaged, not assigned back
    di_2 = di[2]  # unmanaged...
    di_2[0] += 1
    di[2] = di_2  # ... but assigned back
    di[3][0] += 1 # managed, direct manipulation

if __name__ == '__main__':
    manager = multiprocessing.Manager()
    ns = manager.Namespace()
    ns.x = 1
    ns.y = [1]
    ns.z = [1]
    ls = manager.list([1, [1], [1], manager.list([1])])
    di = manager.dict({0: 1, 1: [1], 2: [1], 3: manager.list([1])})

    print('before', ns, ls, ls[2], di, di[2], sep='\n')
    p = multiprocessing.Process(target=f, args=(ns, ls, di))
    p.start()
    p.join()
    print('after', ns, ls, ls[2], di, di[2], sep='\n')

输出:

before
Namespace(x=1, y=[1], z=[1])
[1, [1], [1], <ListProxy object, typeid 'list' at 0x10b8c4630>]
[1]
{0: 1, 1: [1], 2: [1], 3: <ListProxy object, typeid 'list' at 0x10b8c4978>}
[1]
after
Namespace(x=2, y=[1], z=[2])
[2, [1], [2], <ListProxy object, typeid 'list' at 0x10b8c4630>]
[2]
{0: 2, 1: [1], 2: [2], 3: <ListProxy object, typeid 'list' at 0x10b8c4978>}
[2]

如您所见，当一个新值直接分配给托管容器时，它会发生变化；当它被分配给托管容器内的可变容器时，它不会改变；但是如果可变容器被重新分配给托管容器，它会再次改变.使用嵌套的托管容器也可以，直接检测更改而无需重新分配给父容器.

As you can see, when a new value is assigned directly to the managed container, it changes; when it is assigned to a mutable container within the managed container, it doesn't change; but if the mutable container is then reassigned to the managed container, it changes again. Using a nested managed container also works, detecting changes directly without having to assign back to the parent container.

这篇关于如何使用 multiprocessing.Manager()?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！