Python多重处理，传递对象引用包含信号量 [英] Python multiprocessing, passing an object reference containig a semaphore

问题描述

我有这样的情况: 我创建了一个包含信号量的class元素的对象.

I've a scenario like this: I've created an object of the class element containing a semaphore.

import multiprocessing as mpr

class Element(object):
    def __init__(self):
        self.sem = mpr.Semaphore()
        self.xyz = 33

def fun( ch ):
    a = ch.recv()
    print( a[0] )
    print( a[1].xyz )
    a[1].xyz = 99
    print( a[1].xyz )


el = Element()

( pa , ch ) = mpr.Pipe()
proc = mpr.Process(target=fun , args=( ch, ) )

proc.start()
pa.send( [ "Hallo" , el ])

print( el.xyz )

proc.join()

此代码返回此错误:

  File "/usr/lib/python2.7/multiprocessing/forking.py", line 51, in assert_spawning
    ' through inheritance' % type(self).__name__
RuntimeError: Semaphore objects should only be shared between processes through inheritance

但是，如果我从Element的声明中删除信号量，则代码可以工作，但是分配给a [1] .xyz的值将丢失.

But if I remove the semaphore from the declaration of Element the code works, but the value assigned to a[1].xyz will be lost.

现在，我需要通过semphore和多处理来同步大量对象. 因此，有某种方法可以在每个对象中设置信号量，并仅将引用传递给主对象?

Now I need to synchronizes a big collection of object via semphore and multiprocessing. So there's some method for setting a semaphore in every object and passing only the reference to the main object?

import multiprocessing as mpr

class Element(object):
    def __init__(self):
        self.xyz = 33

def fun( ch ):
    a = ch.recv()
    print( a[0] )
    print( a[1].xyz )
    a[1].xyz = 99
    print( a[1].xyz )


el = Element()

( pa , ch ) = mpr.Pipe()
proc = mpr.Process(target=fun , args=( ch, ) )

proc.start()
pa.send( [ "Hallo" , el ])

print( el.xyz )

proc.join()

第二个版本不会产生任何错误，但是分配给a[1].xyz = 99的值将在主进程中丢失.

The second version dot't produce any error, but the value assigned to a[1].xyz = 99 will be lost in the main process.

推荐答案

我认为您不了解multiprocessing模块的工作原理.

I don't think you understood how the multiprocessing module works.

通过管道发送某些内容时，该内容将被腌制，然后在子流程中被取消腌制. 这意味着子进程实际上具有原始对象的副本！ 这就是为什么更改丢失"的原因.添加信号灯不会改变任何内容.

When you send something through the pipe, it gets pickled and then unpickled in the subprocess. This means that the subprocess actually has a copy of the original object! That's why the change is "lost". Adding a semaphore wont change anything.

如果您想要共享内存中的对象，则应使用 multiprocessing.Value ，即使它不处理任意类型. 您可能正在寻找 multiprocessing.Manager .

If you want an object in shared memory you should use multiprocessing.Value, even though this does not handle arbitrary types. Probably multiprocessing.Manager is what you are looking for.

另一种方法是将响应发送到提供修改后的对象的主进程.

An other way would be to send a response to the main process providing the modified object.

这篇关于Python多重处理，传递对象引用包含信号量的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！