python multiprocessing manager列表错误:[Errno 2]没有这样的文件或目录 [英] python multiprocessing manager list error: [Errno 2] No such file or directory
问题描述
我在python中编写了一个多处理程序.我使用multiprocessing.Manager().list()
共享子流程中的列表.首先,我在主要流程中添加了一些任务.然后,启动一些子流程来执行在共享列表中的任务,这些子流程还将任务添加到共享列表中.但是我有一个例外,如下所示:
I worte a multiprocessing program in python. I use multiprocessing.Manager().list()
to share list within subprocess. At first, I add some tasks in main process. And then, start some subprocesses to do tasks which in the shared list, the subprocesses also add tasks to the shared list. But I got a exception as follow:
Traceback (most recent call last):
File "/usr/lib64/python2.6/multiprocessing/process.py", line 232, in _bootstrap
self.run()
File "/usr/lib64/python2.6/multiprocessing/process.py", line 88, in run
self._target(*self._args, **self._kwargs)
File "gen_friendship.py", line 255, in worker
if tmpu in nodes:
File "<string>", line 2, in __contains__
File "/usr/lib64/python2.6/multiprocessing/managers.py", line 722, in _callmethod
self._connect()
File "/usr/lib64/python2.6/multiprocessing/managers.py", line 709, in _connect
conn = self._Client(self._token.address, authkey=self._authkey)
File "/usr/lib64/python2.6/multiprocessing/connection.py", line 143, in Client
c = SocketClient(address)
File "/usr/lib64/python2.6/multiprocessing/connection.py", line 263, in SocketClient
s.connect(address)
File "<string>", line 1, in connect
error: [Errno 2] No such file or directory
我发现了一些有关如何在python多重处理中使用共享列表的信息,例如
I find something about how to use shared list in python multiprocessing like this. But still have some exception. I have no idea of the meaning of the exception. And what's the difference between the common list and the manager.list?
代码如下:
nodes = multiprocessing.Manager().list()
lock = multiprocessing.Lock()
AMOUNT_OF_PROCESS = 10
def worker():
lock.acquire()
nodes.append(node)
lock.release()
if __name__ == "__main__":
for i in range(i):
nodes.append({"name":"username", "group":1})
processes = [None for i in range(AMOUNT_OF_PROCESS)]
for i in range(AMOUNT_OF_PROCESS):
processes[i] = multiprocessing.Process(taget=worker, args=())
processes[i].start()
推荐答案
问题是启动所有辅助进程后,主进程将立即退出,这将关闭Manager
.当您的Manager
关闭时,没有任何孩子可以使用您传递给他们的共享列表.您可以通过使用join
等待所有子项完成来修复它.只需确保在调用join
之前实际上已经start
所有进程:
The problem is that your main process is exiting immediately after you start all your worker processes, which shuts down your Manager
. When your Manager
shuts down, none of the children can use the shared list you passed into them. You can fix it by using join
to wait for all the children to finish. Just make sure you actually start
all your processes prior to calling join
:
for i in range(AMOUNT_OF_PROCESS):
processes[i] = multiprocessing.Process(taget=worker, args=())
processes[i].start()
for process in processes:
process.join()
这篇关于python multiprocessing manager列表错误:[Errno 2]没有这样的文件或目录的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!