Python多处理过程中的错误 [英] Error in Python multiprocessing process

问题描述

我正在尝试写一个包含多个进程的Python代码，其结构和流程如下：

I am trying a write a python code having multiple processes whose structure and flow is something like this:

import multiprocessing
import ctypes
import time
import errno
m=multiprocessing.Manager()
mylist=m.list()
var1=m.Value('i',0)
var2=m.Value('i',1)
var3=m.Value('i',2)
var4=m.Value(ctypes.c_char_p,"a")
var5=m.Value(ctypes.c_char_p,"b")
var6=3
var7=4
var8=5
var9=6
var10=7
def func(var1,var2,var4,var5,mylist):
    i=0
    try:
        if var1.value==0:
            print var2.value,var4.value,var5.value
            mylist.append(time.time())
        elif var1.value==1:
            i=i+2
            print var2.value+2,var4.value,var5.value
            mylist.append(time.time())
    except IOError as e:
        if e.errno==errno.EPIPE:
            var3.value=var3.value+1
            print "Error"
def work():
    for i in range(var3.value):
        print i,var6,var7,va8,var9,var10
        p=multiprocessing.Process(target=func,args=(var1,var2,var4,var5,mylist))
        p.start()
work()

当我运行这个代码时，没有运行精确的循环计数，有时我得到以下错误：

When I run this code, sometimes it works perfectly, sometimes it does not run for exact amount of loop counts and sometimes I get following error:

0
1
Process Process-2:
Traceback (most recent call last):
  File "/usr/lib64/python2.6/multiprocessing/process.py", line 232, in _bootstrap
    self.run()
  File "/usr/lib64/python2.6/multiprocessing/process.py", line 88, in run
    self._target(*self._args, **self._kwargs)
  File "dummy.py", line 19, in func
    if var1.value==0:
  File "/usr/lib64/python2.6/multiprocessing/managers.py", line 1005, in get
    return self._callmethod('get')
  File "/usr/lib64/python2.6/multiprocessing/managers.py", line 722, in _callmethod
    self._connect()
  File "/usr/lib64/python2.6/multiprocessing/managers.py", line 709, in _connect
    conn = self._Client(self._token.address, authkey=self._authkey)
  File "/usr/lib64/python2.6/multiprocessing/connection.py", line 149, in Client
    answer_challenge(c, authkey)
  File "/usr/lib64/python2.6/multiprocessing/connection.py", line 383, in answer_challenge
    message = connection.recv_bytes(256)         # reject large message
EOFError

此错误是什么意思？我在这里做错了什么？这个错误表示什么？请引导我走正确的道路。我使用CentOS 6.5

What does this error mean? What wrong am I doing here? What this error indicates? Kindly guide me to the correct path. I am using CentOS 6.5

推荐答案

在多处理中使用共享变量是棘手的。由于python全局解释器锁（GIL），在Python中不能直接实现多处理。当您使用 multiprocessing 模块时，您可以在不同的进程上启动多个任务，但是不能共享内存。
在你的情况下，你需要这个，所以你试图使用共享内存。但是，这里发生的是，你有几个进程试图同时读取相同的内存。为了避免内存损坏，进程锁定当前正在读取的内存地址，禁止其他进程访问它，直到它完成读取。
这里有3个进程试图在你的 if 循环的第一个中评价 var1.value c $ c> func ：第一个进程读取值，其他进程被阻塞，引发错误。
为了避免这种机制，你应该总是自己管理共享变量的 Lock 。
可以尝试使用语法：

Working with shared variables in multiprocessing is tricky. Because of the python Global Interpreter Lock (GIL), multiprocessing is not directly possible in Python. When you use the multiprocessing module, you can launch several task on different process, BUT you can't share the memory. In you case, you need this so you try to use shared memory. But what happens here is that you have several processes trying to read the same memory at the same time. To avoid memory corruption, a process lock the memory address it is currently reading, forbidding other processes to access it until it finishes reading. Here you have 3 processes trying to evaluate var1.value in the first if loop of your func : the first process read the value, and the other are blocked, raising an error. To avoid this mechanism, you should always manage the Lock of your shared variables yourself. You can try with syntax:

var1=multiprocessing.Value('i',0) # create shared variable
var1.acquire() # get the lock : it will wait until lock is available
var1.value # read the value
var1.release() # release the lock

外部文档：

锁定： https://docs.python.org/2/librar/multiprocessing.html#synchronization-between -processes
GIL： https：/ /docs.python.org/2/glossary.html#term-global-interpreter-lock

这篇关于Python多处理过程中的错误的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！