Python 多处理:按值传递的对象? [英] Python multiprocessing: object passed by value?

问题描述

我一直在尝试以下方法:

I have been trying the following:

from multiprocessing import Pool

def f(some_list):
    some_list.append(4)
    print 'Child process: new list = ' + str(some_list)
    return True

if __name__ == '__main__':

    my_list = [1, 2, 3]
    pool = Pool(processes=4)
    result = pool.apply_async(f, [my_list])
    result.get()

    print 'Parent process: new list = ' + str(my_list)

我得到的是:

Child process: new list = [1, 2, 3, 4]
Parent process: new list = [1, 2, 3]

所以，这意味着 my_list 是按值传递的，因为它没有发生变异.那么，传递给另一个进程时，真的是按值传递的规则吗?谢谢.

So, it means that the my_list was passed by value since it did not mutate. So, is the rule that it is really passed by value when passed to another process? Thanks.

推荐答案

正如 André Laszlo 所说，multiprocessing 库需要pickle 传递给multiprocessing.Pool 方法的所有对象以便将它们传递给工作进程.酸洗过程会导致在工作进程中创建一个不同的对象，因此在工作进程中对对象所做的更改对父进程中的对象没有影响.在 Linux 上，对象有时会通过 fork 继承传递给子进程(例如 multiprocessing.Process(target=func, args=(my_list,)))，但在那种情况下如果您最终在子进程中得到对象的写时复制版本，那么当您尝试在任一进程中修改它时，您仍然会得到不同的副本.

As André Laszlo said, the multiprocessing library needs to pickle all objects passed to multiprocessing.Pool methods in order to pass them to worker processes. The pickling process results in a distinct object being created in the worker process, so that changes made to the object in the worker process have no effect on the object in the parent. On Linux, objects sometimes get passed to the child process via fork inheritence (e.g. multiprocessing.Process(target=func, args=(my_list,))), but in that case you end up with a copy-on-write version of the object in the child process, so you still end up with distinct copies when you try to modify it in either process.

如果你想在进程之间共享一个对象，你可以使用 <代码>multiprocessing.Manager:

If you do want to share an object between processes, you can use a multiprocessing.Manager for that:

from multiprocessing import Pool, Manager

def f(some_list):
    some_list.append(4)
    print 'Child process: new list = ' + str(some_list)
    return True

if __name__ == '__main__':

    my_list = [1, 2, 3]
    m = Manager()
    my_shared_list = m.list(my_list)
    pool = Pool(processes=4)
    result = pool.apply_async(f, [my_shared_list])
    result.get()

    print 'Parent process: new list = ' + str(my_shared_list)

输出:

Child process: new list = [1, 2, 3, 4]
Parent process: new list = [1, 2, 3, 4]

这篇关于Python 多处理:按值传递的对象?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！