python的多处理中的共享变量 [英] Shared variable in python's multiprocessing

问题描述

第一个问题是Value和Manager().Value有什么区别?

First question is what is the difference between Value and Manager().Value?

第二，是否可以在不使用Value的情况下共享整数变量? 下面是我的示例代码.我想要的是获取一个具有整数值而不是Value的字典.我所做的就是在此过程之后将其全部更改.有没有更简单的方法?

Second, is it possible to share integer variable without using Value? Below is my sample code. What I want is getting a dict with a value of integer, not Value. What I did is just change it all after the process. Is there any easier way?

from multiprocessing import Process, Manager

def f(n):
    n.value += 1

if __name__ == '__main__':
    d = {}
    p = []

    for i in range(5):
        d[i] = Manager().Value('i',0)
        p.append(Process(target=f, args=(d[i],)))
        p[i].start()

    for q in p:
        q.join()

    for i in d:
        d[i] = d[i].value

    print d

推荐答案

使用 ctypes 对象在共享内存中，默认情况下使用 RLock 进行同步.当您使用 Manager 时， SynManager 对象，该对象控制允许以下操作的服务器进程要由其他进程操纵的对象值.您可以使用同一管理器创建多个代理；无需在循环中创建新的经理:

When you use Value you get a ctypes object in shared memory that by default is synchronized using RLock. When you use Manager you get a SynManager object that controls a server process which allows object values to be manipulated by other processes. You can create multiple proxies using the same manager; there is no need to create a new manager in your loop:

manager = Manager()
for i in range(5):
    new_value = manager.Value('i', 0)

Manager可以在计算机之间共享，而Value限于一台计算机. Value会更快(运行下面的代码查看)，所以我认为除非您需要支持任意对象或通过网络访问它们，否则您应该使用它.

The Manager can be shared across computers, while Value is limited to one computer. Value will be faster (run the below code to see), so I think you should use that unless you need to support arbitrary objects or access them over a network.

import time
from multiprocessing import Process, Manager, Value

def foo(data, name=''):
    print type(data), data.value, name
    data.value += 1

if __name__ == "__main__":
    manager = Manager()
    x = manager.Value('i', 0)
    y = Value('i', 0)

    for i in range(5):
        Process(target=foo, args=(x, 'x')).start()
        Process(target=foo, args=(y, 'y')).start()

    print 'Before waiting: '
    print 'x = {0}'.format(x.value)
    print 'y = {0}'.format(y.value)

    time.sleep(5.0)
    print 'After waiting: '
    print 'x = {0}'.format(x.value)
    print 'y = {0}'.format(y.value)

总结:

1. 使用Manager创建多个共享对象，包括字典和 列表.使用Manager在网络上的计算机之间共享数据.
2. 在不需要共享信息时使用Value或Array 跨网络，ctypes中的类型足以满足您的需求 需求.
3. Value比Manager快.

1. Use Manager to create multiple shared objects, including dicts and lists. Use Manager to share data across computers on a network.
2. Use Value or Array when it is not necessary to share information across a network and the types in ctypes are sufficient for your needs.
3. Value is faster than Manager.

警告

顺便说一句，如果可能的话，应该避免跨进程/线程共享数据.上面的代码可能会按预期运行，但是会增加执行foo所花费的时间，并且事情会变得很奇怪.将以上内容与:

By the way, sharing data across processes/threads should be avoided if possible. The code above will probably run as expected, but increase the time it takes to execute foo and things will get weird. Compare the above with:

def foo(data, name=''):
    print type(data), data.value, name
    for j in range(1000):
        data.value += 1

您需要Lock才能使其正常工作.

You'll need a Lock to make this work correctly.

我不太了解所有这一切，因此也许其他人会出现并提供更多见解.我想我会提供一个答案，因为这个问题没有引起人们的注意.希望能有所帮助.

I am not especially knowledgable about all of this, so maybe someone else will come along and offer more insight. I figured I would contribute an answer since the question was not getting attention. Hope that helps a little.

这篇关于python的多处理中的共享变量的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！