使用共享内存进行多处理 [英] Multiprocessing using shared memory
问题描述
有人可以给我提供示例代码,以便在使用锁的python多处理模块中,在一组工作进程甚至单个生成的进程之间共享可写数组或列表吗?我下面的代码产生了2个进程,其中一个应向共享数组打印"1",另一个应向共享数组打印"2".但是,当我尝试在处理后打印出数组的元素时,它只会显示0列表.我要去哪里错了?我希望在多个进程之间共享可写的数据结构.
Can someone provide me with sample code to share a writable array or a list among a pool of worker processes or even individually spawned processes using the multiprocessing module of python using locks? My code below spawns 2 processes of which one should print '1' and the other should print '2' to the shared array. However, when I try to print out the elements of the array after the processing it only gives me a list of 0's. Where am I going wrong? I want a writable data structure to be shared between multiple processes.
下面是我的代码:
import multiprocessing
arr=multiprocessing.Array('i',10,lock=True)
lock=multiprocessing.RLock()
def putitin(n):
for i in range(5):
lock.acquire()
arr.append(n)
lock.release()
return
p1=multiprocessing.Process(target=putitin,args=(1,))
p2=multiprocessing.Process(target=putitin,args=(2,))
p1.start()
p2.start()
p1.join()
p2.join()
for i in range(10):
print(arr[i])
推荐答案
代码的一个潜在问题是,要在Windows上使用multiprocessing
,您需要将主进程的代码放在if __name__ == '__main__':
块中.请参见 Windows < 多处理编程指南 .
One potential problem with your code is that to use multiprocessing
on Windows you need to put the code for the main process in an if __name__ == '__main__':
block. See the Safe importing of main module subsection of the Windows section of the multiprocessing Programming guidelines.
另一个主要问题是您正在尝试在各个进程之间共享全局变量.这是行不通的,因为每个进程都在自己的未共享内存空间中运行,因此每个子进程都有自己的arr
. (不过,只是模块级常量的变量还是可以的)
Another major one is that you're attempting to a share global variable among the processes. which won't work because each one runs in its own unshared memory-space, so each subprocess has it's own arr
. (Variables which are just module level constants are OK, though)
最后,multiprocessing.Array
具有固定的大小,并且没有代码试图在putitin()
函数中使用的extend()
方法-因此,看来您还想要一个可写的和可调整大小的容器(已排序,并且可以通过整数索引访问).
Lastly, a multiprocessing.Array
has a fixed size and doesn't have the extend()
method your code is trying to use in the putitin()
function — therefore it appears you also want a writable and resizable container (that is ordered and perhaps accessible via integer indices).
在这种情况下,类似以下的内容可能是合适的.您不需要在更改对象之前显式锁定对象,因为它是线程安全的共享对象.
In that case something the like the following may be suitable. You don't need to explicitly lock the object before make changes to it because it's a thread-safe shared object.
import multiprocessing
def putitin(lst, value):
for i in range(5):
lst.append(value)
if __name__ == '__main__':
manager = multiprocessing.Manager()
lst = manager.list()
p1 = multiprocessing.Process(target=putitin, args=(lst, 1))
p2 = multiprocessing.Process(target=putitin, args=(lst, 2))
p1.start()
p2.start()
p1.join()
p2.join()
for i in range(len(lst)):
print(lst[i])
输出:
1
1
1
1
1
2
2
2
2
2
这篇关于使用共享内存进行多处理的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!