在PyQt5应用程序中，可以并行运行sklearn而不冻结 [英] In a PyQt5 application, is it possible to run sklearn with parallel jobs without freezing

问题描述

是否可以在qt应用程序中运行而不冻结gui，例如

Is it possible to run, in a qt application, without freezing the gui, let's say a sklearn gird search that use several jobs parallel (n_jobs > 1)? The problem is that joblib that is used for parallelizing sklearn code cannot run multiprocess into a thread.

例如，我正在使用Gridsearch来找到svr的最佳参数，这是相当计算密集的.

For example, I'm using Gridsearch to find the best parameters for a svr, which is quite computionnaly intensive.

这个问题已经问了好几次了，但是没有找到解决方法:

This question has been asked several times, but no solution found:

pyqt5-run-sklearn-calculations-on-a-separate -qthread ，建议使用QProcess吗?

multiprocessing-backed-parallel-loops-cannot-be -nested-below-threads ，threading.current_thread().name = 'MainThread'解决方法在问题后不起作用已修复

joblib-parallel-uses-only-one -core-if-started-from-qthread ，使用multiprocessing.Pool(processes=4)重写任务.此方法不适用于gridsearch嵌入njobs.

joblib-parallel-uses-only-one-core-if-started-from-qthread, rewrite the task using multiprocessing.Pool(processes=4). This method is not applicable for gridsearch embed njobs.

使用sklearn交叉验证序列，在PyQt按钮中，没有其他要求...

use sklearn cross validation train, in PyQt button, No aswers...

以及不支持为何有目的地的任何见解(

And any insight why this is purposely not supported (it a feature) ? It seems like it something that would be quite useful ?

推荐答案

根据我对问题的了解，问题出在joblib使用的默认后端，即loky.

From my understanding of the issue, the problem resides with the default backend used by joblib, namely loky.

经过 joblib 和

After some digging through the joblib and sklearn documentation, I resolved my issue by switching the joblib backend to threading. Note, the call to register_parallel_backend lies outside the __init__ function.

from sklearn.utils import parallel_backend, register_parallel_backend
from joblib._parallel_backends import ThreadingBackend

class ModelTrainer(QRunnable):
    register_parallel_backend('threading', ThreadingBackend, make_default=True)

    def __init__(self, **kwargs):

这篇关于在PyQt5应用程序中，可以并行运行sklearn而不冻结的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！