python multiprocessing-将子进程日志记录发送到在父级中运行的GUI [英] python multiprocessing - sending child process logging to GUI running in parent
问题描述
我正在我编写的一些分析代码之上构建一个接口，该接口执行一些SQL并处理查询结果.我想向用户展示此分析代码中许多事件的日志记录.因为分析代码运行时间很长，并且因为我不想让UI阻塞，所以到目前为止，我已经通过将分析函数放入其自己的线程中来完成此操作.
I'm building an interface on top of some analysis code I've written that executes some SQL and processes the query results. There's logging surrounding a number of the events in this analysis code that I would like to expose to the user. Because the analysis code is rather long-running, and because I don't want the UI to block, thus far I've done this through putting the analysis function in to its own thread.
我现在拥有的简化示例(完整的脚本):
Simplified example of what I have now (complete script):
import sys
import time
import logging
from PySide2 import QtCore, QtWidgets
def long_task():
logging.info('Starting long task')
time.sleep(3) # this would be replaced with a real task
logging.info('Long task complete')
class LogEmitter(QtCore.QObject):
sigLog = QtCore.Signal(str)
class LogHandler(logging.Handler):
def __init__(self):
super().__init__()
self.emitter = LogEmitter()
def emit(self, record):
msg = self.format(record)
self.emitter.sigLog.emit(msg)
class LogDialog(QtWidgets.QDialog):
def __init__(self, parent=None):
super().__init__(parent)
log_txt = QtWidgets.QPlainTextEdit(self)
log_txt.setReadOnly(True)
layout = QtWidgets.QHBoxLayout(self)
layout.addWidget(log_txt)
self.setWindowTitle('Event Log')
handler = LogHandler()
handler.emitter.sigLog.connect(log_txt.appendPlainText)
logger = logging.getLogger()
logger.addHandler(handler)
logger.setLevel(logging.INFO)
class Worker(QtCore.QThread):
results = QtCore.Signal(object)
def __init__(self, func, *args, **kwargs):
super().__init__()
self.func = func
self.args = args
self.kwargs = kwargs
def run(self):
results = self.func(*self.args, **self.kwargs)
self.results.emit(results)
class MainWindow(QtWidgets.QMainWindow):
def __init__(self):
super().__init__()
widget = QtWidgets.QWidget()
layout = QtWidgets.QHBoxLayout(widget)
start_btn = QtWidgets.QPushButton('Start')
start_btn.clicked.connect(self.start)
layout.addWidget(start_btn)
self.setCentralWidget(widget)
self.log_dialog = LogDialog()
self.worker = None
def start(self):
if not self.worker:
self.log_dialog.show()
logging.info('Run Starting')
self.worker = Worker(long_task)
self.worker.results.connect(self.handle_result)
self.worker.start()
def handle_result(self, result=None):
logging.info('Result received')
self.worker = None
if __name__ == '__main__':
app = QtWidgets.QApplication()
win = MainWindow()
win.show()
sys.exit(app.exec_())
这很好，除了我需要能够允许用户停止执行分析代码.我读过的所有内容都表明没有办法很好地中断线程，因此使用multiprocessing库似乎是可行的方法(没有办法重新编写分析代码以允许定期轮询，因为大多数情况下，只是等待查询返回结果而花费的时间).通过使用multiprocessing.Pool和apply_async不会阻塞UI的方式执行分析代码，获得相同的功能非常容易.
This works fine, except that I need to be able to allow the user to stop the execution of the analysis code. Everything I've read indicates that there is no way to interrupt threads nicely, so using the multiprocessing library seems to be the way to go (there's no way to re-write the analysis code to allow for periodic polling, since the majority of time is spent just waiting for the queries to return results). It's easy enough to get the same functionality in terms of executing the analysis code in a way that doesn't block the UI by using multiprocessing.Pool and apply_async.
例如从上方将MainWindow替换为:
class MainWindow(QtWidgets.QMainWindow):
def __init__(self):
super().__init__()
widget = QtWidgets.QWidget()
layout = QtWidgets.QHBoxLayout(widget)
start_btn = QtWidgets.QPushButton('Start')
start_btn.clicked.connect(self.start)
layout.addWidget(start_btn)
self.setCentralWidget(widget)
self.log_dialog = LogDialog()
self.pool = multiprocessing.Pool()
self.running = False
def start(self):
if not self.running:
self.log_dialog.show()
logging.info('Run Starting')
self.pool.apply_async(long_task, callback=self.handle_result)
def handle_result(self, result=None):
logging.info('Result received')
self.running = False
但是我似乎无法弄清楚如何从子进程中检索日志输出并将其传递给父进程以更新日志对话框.我已经阅读了有关此问题的每一个SO问题，以及有关如何处理从多个进程写入单个日志文件的食谱示例，但是我无法全神贯注于如何使这些想法适应我的想法.我想在这里做.
But I can't seem to figure out how I would go about retrieving the logging output from the child process and passing it to the parent to update the log dialog. I've read through just about every SO question on this as well as the cookbook examples of how to handle writing to a single log file from multiple processes, but I can't wrap my head around how to adapt those ideas to what I'm trying to do here.
所以试图弄清楚为什么我看到的行为与我添加的@eyllanesc不同:
So trying to figure out what might be going on for why I'm seeing different behavior than @eyllanesc I added:
logger = logging.getLogger()
print(f'In Func: {logger} at {id(logger)}')
和
logger = logging.getLogger()
print(f'In Main: {logger} at {id(logger)}')
分别
到long_task和Mainwindow.start.当我运行main.py时，我得到:
to long_task and Mainwindow.start, respectively. When I run main.py I get:
In Main: <RootLogger root (INFO)> at 2716746681984
In Func: <RootLogger root (WARNING)> at 1918342302352
这似乎是中描述的问题
使用Queue和QueueHandler作为解决方案的想法似乎类似于@eyllanesc的原始解决方案
This idea of using a Queue and QueueHandler though as a solution seems similar to @eyllanesc's original solution
推荐答案
信号不会在进程之间传输数据，因此在这种情况下，必须使用Pipe然后发出信号:
The signals do not transmit data between processes, so for this case a Pipe must be used and then emit the signal:
# other imports
import threading
# ...
class LogHandler(logging.Handler):
def __init__(self):
super().__init__()
self.r, self.w = multiprocessing.Pipe()
self.emitter = LogEmitter()
threading.Thread(target=self.listen, daemon=True).start()
def emit(self, record):
msg = self.format(record)
self.w.send(msg)
def listen(self):
while True:
try:
msg = self.r.recv()
self.emitter.sigLog.emit(msg)
except EOFError:
break
# ...
这篇关于python multiprocessing-将子进程日志记录发送到在父级中运行的GUI的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
