Python:为什么这里的进程之间共享多处理锁? [英] Python: Why is the multiprocessing lock shared among processes here?

问题描述

我正在尝试在进程之间共享锁.我知道共享锁的方法是将其作为参数传递给目标函数.但是，我发现即使是下面的方法也有效.我无法理解进程共享此锁的方式.有人可以解释一下吗?

I am trying to share a lock among processes. I understand that the way to share a lock is to pass it as an argument to the target function. However I found that even the approach below is working. I could not understand the way the processes are sharing this lock. Could anyone please explain?

import multiprocessing as mp
import time


class SampleClass:

    def __init__(self):
        self.lock = mp.Lock()
        self.jobs = []
        self.total_jobs = 10

    def test_run(self):
        for i in range(self.total_jobs):
            p = mp.Process(target=self.run_job, args=(i,))
            p.start()
            self.jobs.append(p)

        for p in self.jobs:
            p.join()

    def run_job(self, i):
        with self.lock:
            print('Sleeping in process {}'.format(i))
            time.sleep(5)


if __name__ == '__main__':
    t = SampleClass()
    t.test_run()

推荐答案

在Windows(您说过，您正在使用)上，这些事情总是会简化为有关 multiprocessing 如何与 pickle ，因为Windows上所有Python数据跨越进程的边界都是通过在发送端进行酸洗(在接收端进行酸洗)来实现的.

On Windows (which you said you're using), these kinds of things always reduce to details about how multiprocessing plays with pickle, because all Python data crossing process boundaries on Windows is implemented by pickling on the sending end (and unpickling on the receiving end).

我最好的建议是避免做引起此类问题的开头；-)例如，您显示的代码在Windows下使用Python 2时会崩溃，而在Python 3下使用时也会崩溃.multiprocessing.Pool 方法，而不是 multiprocessing.Process .

My best advice is to avoid doing things that raise such questions to begin with ;-) For example, the code you showed blows up on Windows under Python 2, and also blows up under Python 3 if you use a multiprocessing.Pool method instead of multiprocessing.Process.

这不仅仅是锁，只是尝试腌制绑定方法(例如 self.run_job )在Python 2中会炸毁.请考虑一下.您正在跨越过程边界，并且在接收端没有 对应于 self 的对象. self.run_job 应该绑定到接收端的哪个对象?

It's not just the lock, simply trying to pickle a bound method (like self.run_job) blows up in Python 2. Think about it. You're crossing a process boundary, and there isn't an object corresponding to self on the receiving end. To what object is self.run_job supposed to be bound on the receiving end?

在Python 3中，腌制 self.run_job 也腌制 self 对象的副本.答案就是这样:魔术在接收端创建了一个与 self 相对应的 SampleClass 对象.清澈如泥.腌制 t 的整个状态，包括 t.lock .这就是为什么它起作用"的原因.

In Python 3, pickling self.run_job also pickles a copy of the self object. So that's the answer: a SampleClass object corresponding to self is created by magic on the receiving end. Clear as mud. t's entire state is pickled, including t.lock. That's why it "works".

有关更多实施细节，请参见此:

See this for more implementation details:

为什么我可以将实例方法传递给multiprocessing.Process，而不传递给multiprocessing.Pool吗?

从长远来看，如果您坚持那些显然可以工作的事情，那么将蒙受最少的困扰:传递模块全局可调用对象(既不提供实例方法也不提供本地函数)，并显式传递多处理数据对象(是否为 Lock ， Queue ， manager.list 等的实例).

In the long run, you'll suffer the fewest mysteries if you stick to things that were obviously intended to work: pass module-global callable objects (neither, e.g., instance methods nor local functions), and explicitly pass multiprocessing data objects (whether an instance of Lock, Queue, manager.list, etc etc).

这篇关于Python:为什么这里的进程之间共享多处理锁?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！