python多处理中的共享变量 [英] Shared variable in python's multiprocessing

问题描述

第一个问题是Value和Manager().Value有什么区别?

First question is what is the difference between Value and Manager().Value?

其次，是否可以不使用 Value 共享整数变量?下面是我的示例代码.我想要的是得到一个整数值的字典，而不是值.我所做的只是在这个过程之后改变它.有没有更简单的方法?

Second, is it possible to share integer variable without using Value? Below is my sample code. What I want is getting a dict with a value of integer, not Value. What I did is just change it all after the process. Is there any easier way?

from multiprocessing import Process, Manager

def f(n):
    n.value += 1

if __name__ == '__main__':
    d = {}
    p = []

    for i in range(5):
        d[i] = Manager().Value('i',0)
        p.append(Process(target=f, args=(d[i],)))
        p[i].start()

    for q in p:
        q.join()

    for i in d:
        d[i] = d[i].value

    print d

推荐答案

当你使用 Value 你会得到一个 共享内存中的 ctypes 对象，默认情况下使用 RLock.当您使用 Manager 你会得到一个 SynManager 控制服务器进程的对象，该服务器进程允许其他进程操作对象值.您可以使用同一个管理器创建多个代理；无需在循环中创建新管理器:

When you use Value you get a ctypes object in shared memory that by default is synchronized using RLock. When you use Manager you get a SynManager object that controls a server process which allows object values to be manipulated by other processes. You can create multiple proxies using the same manager; there is no need to create a new manager in your loop:

manager = Manager()
for i in range(5):
    new_value = manager.Value('i', 0)

Manager 可以跨计算机共享，而Value 仅限于一台计算机.Value 会更快(运行下面的代码查看)，所以我认为你应该使用它，除非你需要支持任意对象或通过网络访问它们.

The Manager can be shared across computers, while Value is limited to one computer. Value will be faster (run the below code to see), so I think you should use that unless you need to support arbitrary objects or access them over a network.

import time
from multiprocessing import Process, Manager, Value

def foo(data, name=''):
    print type(data), data.value, name
    data.value += 1

if __name__ == "__main__":
    manager = Manager()
    x = manager.Value('i', 0)
    y = Value('i', 0)

    for i in range(5):
        Process(target=foo, args=(x, 'x')).start()
        Process(target=foo, args=(y, 'y')).start()

    print 'Before waiting: '
    print 'x = {0}'.format(x.value)
    print 'y = {0}'.format(y.value)

    time.sleep(5.0)
    print 'After waiting: '
    print 'x = {0}'.format(x.value)
    print 'y = {0}'.format(y.value)

总结一下:

1. 使用 Manager 创建多个共享对象，包括字典和列表.使用 Manager 在网络上的计算机之间共享数据.
2. 在不需要共享信息时使用 Value 或 Array通过网络和 ctypes 中的类型足以满足您的需求需要.
3. Value 比 Manager 快.

1. Use Manager to create multiple shared objects, including dicts and lists. Use Manager to share data across computers on a network.
2. Use Value or Array when it is not necessary to share information across a network and the types in ctypes are sufficient for your needs.
3. Value is faster than Manager.

警告

顺便说一句，如果可能，应避免跨进程/线程共享数据.上面的代码可能会按预期运行，但是会增加执行 foo 所需的时间，事情会变得很奇怪.将上述内容与:

By the way, sharing data across processes/threads should be avoided if possible. The code above will probably run as expected, but increase the time it takes to execute foo and things will get weird. Compare the above with:

def foo(data, name=''):
    print type(data), data.value, name
    for j in range(1000):
        data.value += 1

您需要一个 Lock 才能使其正常工作.

You'll need a Lock to make this work correctly.

我对这一切并不是特别了解，所以也许其他人会来提供更多见解.我想我会提供一个答案，因为这个问题没有引起注意.希望对您有所帮助.

I am not especially knowledgable about all of this, so maybe someone else will come along and offer more insight. I figured I would contribute an answer since the question was not getting attention. Hope that helps a little.

这篇关于python多处理中的共享变量的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！