使用atexit终止线程时，脚本卡在出口上 [英] Script stuck on exit when using atexit to terminate threads

问题描述

我正在使用python 3.7.4上的线程，并且我想使用atexit注册一个清理函数，该函数将(干净地)终止线程.

I'm playing around with threads on python 3.7.4, and I want to use atexit to register a cleanup function that will (cleanly) terminate the threads.

例如:

# example.py
import threading
import queue
import atexit
import sys

Terminate = object()

class Worker(threading.Thread):
    def __init__(self):
        super().__init__()
        self.queue = queue.Queue()

    def send_message(self, m):
        self.queue.put_nowait(m)

    def run(self):
        while True:
            m = self.queue.get()
            if m is Terminate:
                break
            else:
                print("Received message: ", m)


def shutdown_threads(threads):
    for t in threads:
        print(f"Terminating thread {t}")
        t.send_message(Terminate)
    for t in threads:
        print(f"Joining on thread {t}")
        t.join()
    else:
        print("All threads terminated")

if __name__ == "__main__":
    threads = [
        Worker()
        for _ in range(5)
    ]
    atexit.register(shutdown_threads, threads)

    for t in threads:
        t.start()

    for t in threads:
        t.send_message("Hello")
        #t.send_message(Terminate)

    sys.exit(0)

但是，似乎与atexit回调中的线程和队列进行交互会创建带有一些内部关闭例程的死锁:

However, it seems interacting with the threads and queues in the atexit callback creates a deadlock with some internal shutdown routine:

$ python example.py
Received message:  Hello
Received message:  Hello
Received message:  Hello
Received message:  Hello
Received message:  Hello
^CException ignored in: <module 'threading' from '/usr/lib64/python3.7/threading.py'>
Traceback (most recent call last):
  File "/usr/lib64/python3.7/threading.py", line 1308, in _shutdown
    lock.acquire()
KeyboardInterrupt
Terminating thread <Worker(Thread-1, started 140612492904192)>
Terminating thread <Worker(Thread-2, started 140612484511488)>
Terminating thread <Worker(Thread-3, started 140612476118784)>
Terminating thread <Worker(Thread-4, started 140612263212800)>
Terminating thread <Worker(Thread-5, started 140612254820096)>
Joining on thread <Worker(Thread-1, stopped 140612492904192)>
Joining on thread <Worker(Thread-2, stopped 140612484511488)>
Joining on thread <Worker(Thread-3, stopped 140612476118784)>
Joining on thread <Worker(Thread-4, stopped 140612263212800)>
Joining on thread <Worker(Thread-5, stopped 140612254820096)>
All threads terminated

(KeyboardInterrupt是我使用ctrl-c的原因，因为该进程似乎无限期挂起).

(the KeyboardInterrupt is me using ctrl-c since the process seems to be hanging indefinitely).

但是，如果我在退出之前发送Terminate消息(取消注释t.send_message("Hello")之后的行)，则程序不会挂起并正常终止:

However, if I send the Terminate message before exit(uncomment the line after t.send_message("Hello")), the program doesn't hang and terminates gracefully:

$ python example.py
Received message:  Hello
Received message:  Hello
Received message:  Hello
Received message:  Hello
Received message:  Hello
Terminating thread <Worker(Thread-1, stopped 140516051592960)>
Terminating thread <Worker(Thread-2, stopped 140516043200256)>
Terminating thread <Worker(Thread-3, stopped 140515961992960)>
Terminating thread <Worker(Thread-4, stopped 140515953600256)>
Terminating thread <Worker(Thread-5, stopped 140515945207552)>
Joining on thread <Worker(Thread-1, stopped 140516051592960)>
Joining on thread <Worker(Thread-2, stopped 140516043200256)>
Joining on thread <Worker(Thread-3, stopped 140515961992960)>
Joining on thread <Worker(Thread-4, stopped 140515953600256)>
Joining on thread <Worker(Thread-5, stopped 140515945207552)>
All threads terminated

这引出了一个问题，相对于atexit处理程序，何时执行此threading._shutdown例程? 在atexit处理程序中与线程进行交互是否有意义?

This begs the question, when does this threading._shutdown routine gets executed, relative to atexit handlers? Does it make sense to interact with threads in atexit handlers?

推荐答案

atexit.register(func)将func注册为要在终止时执行的功能.

atexit.register(func) registers func as a function to be executed at termination.

在主线程中执行最后一行代码(在上面的示例中为sys.exit(0))后，(解释器)调用threading._shutdown以等待所有非守护进程线程(在上述示例中创建的工作程序)退出

After execute the last line of code (it is sys.exit(0) in above example) in main thread, threading._shutdown was invoked (by interpreter) to wait for all non-daemon threads (Workers created in above example) exit

当没有活动的非守护线程时，整个Python程序将退出.

The entire Python program exits when no alive non-daemon threads are left.

因此，在键入 CTRL + C 之后，主线程被SIGINT信号终止，然后解释器调用 atexit 注册函数.

So after typing CTRL+C, the main thread was terminated by SIGINT signal, and then atexit registered functions are called by interpreter.

顺便说一句，如果将daemon=True传递给Thread.__init__，该程序将直接运行，而无需任何人工干预.

By the way, if you pass daemon=True to Thread.__init__, the program would run straightforward without any human interactive.

这篇关于使用atexit终止线程时，脚本卡在出口上的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！