Python - “不能腌制 thread.lock"；Windows多进程下创建线程时出错 [英] Python - "can't pickle thread.lock" error when creating a thread under a multiprocess in Windows

问题描述

我被我认为是基本的多进程和线程问题所困扰.我已经设置了一个多进程，并在其中设置了一个线程.但是，当我在 init 函数中设置线程类时，出现以下错误:

I'm getting stuck on what I think is a basic multiprocess and threading issue. I've got a multiprocess set up, and within this a thread. However, when I set up the thread class within the init function, I get the following error:

类型错误:无法pickle thread.lock 对象".

"TypeError: can't pickle thread.lock objects".

但是，如果线程是在 init 函数之外设置的，则不会发生这种情况.有谁知道为什么会这样?请注意，我使用的是 Windows.

However, this does not happen if the thread is set up outside of the init function. Does anyone know why this is happening? Note I'm using Windows.

下面有一些代码来说明这个问题.如下所示，它运行良好.但是，如果从 DoStuff init def 中调用 print_hello()，则会发生错误，如果在多进程 run() def 中调用它，则没问题.

Some code is below to illustrate the issue. As typed below, it runs fine. However if print_hello() is called from within the DoStuff init def, then the error occurs, if it's called within the multi-process run() def then it's fine.

谁能指出我正确的方向，以便在从 init 调用时运行良好?谢谢！

Can anyone point me in the right direction so it runs fine when called from init? thanks!

import multiprocessing
import threading
import time

class MyProcess(multiprocessing.Process):

    def __init__(self, **kwargs):
        super(MyProcess, self).__init__(**kwargs)
        self.dostuff = DoStuff()

    def run(self):
        print("starting DoStuff")
        # This works fine if the line below is uncommented and __init__ self.print_hello() is commented...
        self.dostuff.print_hello()


class DoStuff(object):
    def __init__(self, **kwargs):
        super(DoStuff, self).__init__(**kwargs)

        # If the following is uncommented, the error occurs...
        #   Note it also occurs if the lines in start_thead are pasted here...
        # self.print_hello()

    def print_hello(self):
        print "hello"
        self.start_thread()

    def start_thread(self):
        self.my_thread_instance = MyThread()
        self.my_thread_instance.start()
        time.sleep(0.1)


class MyThread(threading.Thread):
    def __init__(self):
        super(MyThread, self).__init__()

    def run(self):
        print("Starting MyThread")


if __name__ == '__main__':
    mp_target = MyProcess()       # Also pass the pipe to transfer data
    # mp_target.daemon = True
    mp_target.start()
    time.sleep(0.1)

推荐答案

似乎没有简单的答案，而且似乎是 Windows 的限制(Win 7，在我的情况下为 python 3.6)；在 Windows 上，您似乎需要先启动进程，然后才能在拥有的对象内启动工作线程.

It looks like there is no simple answer, and it appears to be a restriction of Windows (Win 7, python 3.6 in my case); on Windows it looks like you need to start the process before you can start the worker thread inside the owned object.

在 Unix(CentOS 7、python 2.7.5)上似乎没有这样的限制.

There appears to be no such restriction on Unix (CentOS 7, python 2.7.5).

作为实验，我将您的代码修改如下；此版本检查操作系统并首先启动进程或线程:

As an experiment I modified your code as follows; this version checks the OS and starts either the process first, or the thread first:

import multiprocessing
import threading
import time
import os

class MyProcess(multiprocessing.Process):

    def __init__(self, **kwargs):
        super(MyProcess, self).__init__(**kwargs)
        self.dostuff = DoStuff(self)

    def run(self):
        print("MyProcess.run()")
        print("MyProcess.ident = " + repr(self.ident))
        if os.name == 'nt':
            self.dostuff.start_thread()

class DoStuff(object):
    def __init__(self, owner, **kwargs):
        super(DoStuff, self).__init__(**kwargs)
        self.owner = owner
        if os.name != 'nt':
            self.start_thread()

    def start_thread(self):
        print("DoStuff.start_thread()")
        self.my_thread_instance = MyThread(self)
        self.my_thread_instance.start()
        time.sleep(0.1)

class MyThread(threading.Thread):
    def __init__(self, owner):
        super(MyThread, self).__init__()
        self.owner = owner

    def run(self):
        print("MyThread.run()")
        print("MyThread.ident = " + repr(self.ident))
        print("MyThread.owner.owner.ident = " + repr(self.owner.owner.ident))

if __name__ == '__main__':
    mp_target = MyProcess()       # Also pass the pipe to transfer data
    mp_target.daemon = True
    mp_target.start()
    time.sleep(0.1)

... 并在 Windows 上得到以下内容，该过程首先开始:

... and got the following on Windows, where the process starts first:

MyProcess.run()
MyProcess.ident = 14700
DoStuff.start_thread()
MyThread.run() 
MyThread.ident = 14220
MyThread.owner.owner.ident = 14700

... 以及 Linux 上的以下内容，首先启动线程:

... and the following on Linux, where the thread is started first:

DoStuff.start_thread()
MyThread.run()
MyThread.ident = 140316342347520
MyThread.owner.owner.ident = None
MyProcess.run()
MyProcess.ident = 4358

如果是我的代码，我很想总是先启动进程，然后在该进程中创建线程；以下版本在两个平台上都适合我:

If it were my code I'd be tempted to always start the process first, then create the thread within that process; the following version works fine for me across both platforms:

import multiprocessing
import threading
import time

class MyProcess(multiprocessing.Process):

    def __init__(self, **kwargs):
        super(MyProcess, self).__init__(**kwargs)
        self.dostuff = DoStuff()

    def run(self):
        print("MyProcess.run()")
        self.dostuff.start_thread()

class DoStuff(object):
    def __init__(self, **kwargs):
        super(DoStuff, self).__init__(**kwargs)

    def start_thread(self):
        self.my_thread_instance = MyThread()
        self.my_thread_instance.start()
        time.sleep(0.1)

class MyThread(threading.Thread):
    def __init__(self):
        super(MyThread, self).__init__()

    def run(self):
        print("MyThread.run()")

if __name__ == '__main__':
    mp_target = MyProcess()       # Also pass the pipe to transfer data
    mp_target.daemon = True
    mp_target.start()
    time.sleep(0.1)

这篇关于Python - “不能腌制 thread.lock"；Windows多进程下创建线程时出错的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！