使用共享内存进行多处理 [英] Multiprocessing using shared memory

问题描述

有人可以给我提供示例代码，以便在使用锁的python多处理模块中，在一组工作进程甚至单个生成的进程之间共享可写数组或列表吗?我下面的代码产生了2个进程，其中一个应向共享数组打印"1"，另一个应向共享数组打印"2".但是，当我尝试在处理后打印出数组的元素时，它只会显示0列表.我要去哪里错了?我希望在多个进程之间共享可写的数据结构.

Can someone provide me with sample code to share a writable array or a list among a pool of worker processes or even individually spawned processes using the multiprocessing module of python using locks? My code below spawns 2 processes of which one should print '1' and the other should print '2' to the shared array. However, when I try to print out the elements of the array after the processing it only gives me a list of 0's. Where am I going wrong? I want a writable data structure to be shared between multiple processes.

下面是我的代码:

import multiprocessing

arr=multiprocessing.Array('i',10,lock=True)
lock=multiprocessing.RLock()

def putitin(n):
    for i in range(5):
        lock.acquire()
        arr.append(n)
        lock.release()
    return

p1=multiprocessing.Process(target=putitin,args=(1,))
p2=multiprocessing.Process(target=putitin,args=(2,))

p1.start()
p2.start()

p1.join()
p2.join()

for i in range(10):
    print(arr[i])

推荐答案

代码的一个潜在问题是，要在Windows上使用multiprocessing，您需要将主进程的代码放在if __name__ == '__main__':块中.请参见 Windows < 多处理编程指南 .

One potential problem with your code is that to use multiprocessing on Windows you need to put the code for the main process in an if __name__ == '__main__': block. See the Safe importing of main module subsection of the Windows section of the multiprocessing Programming guidelines.

另一个主要问题是您正在尝试在各个进程之间共享全局变量.这是行不通的，因为每个进程都在自己的未共享内存空间中运行，因此每个子进程都有自己的arr. (不过，只是模块级常量的变量还是可以的)

Another major one is that you're attempting to a share global variable among the processes. which won't work because each one runs in its own unshared memory-space, so each subprocess has it's own arr. (Variables which are just module level constants are OK, though)

最后，multiprocessing.Array具有固定的大小，并且没有代码试图在putitin()函数中使用的extend()方法-因此，看来您还想要一个可写的和可调整大小的容器(已排序，并且可以通过整数索引访问).

Lastly, a multiprocessing.Array has a fixed size and doesn't have the extend() method your code is trying to use in the putitin() function — therefore it appears you also want a writable and resizable container (that is ordered and perhaps accessible via integer indices).

在这种情况下，类似以下的内容可能是合适的.您不需要在更改对象之前显式锁定对象，因为它是线程安全的共享对象.

In that case something the like the following may be suitable. You don't need to explicitly lock the object before make changes to it because it's a thread-safe shared object.

import multiprocessing

def putitin(lst, value):
    for i in range(5):
        lst.append(value)

if __name__ == '__main__':
    manager = multiprocessing.Manager()
    lst = manager.list()

    p1 = multiprocessing.Process(target=putitin, args=(lst, 1))
    p2 = multiprocessing.Process(target=putitin, args=(lst, 2))

    p1.start()
    p2.start()

    p1.join()
    p2.join()

    for i in range(len(lst)):
        print(lst[i])

输出:

1
1
1
1
1
2
2
2
2
2

这篇关于使用共享内存进行多处理的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！