线程 QueueManagerThread 中的异常 - scikit-learn [英] Exception in thread QueueManagerThread - scikit-learn

问题描述

当我设置 n_jobs=-1 我得到错误，如果我设置 n_jobs 等于 big value (n_jobs=100)，但是如果设置较小的值(例如 n_jobs=32)，它工作正常.我试过重新安装 scikit-learn 和 joblib 包，但无济于事.此外，它 (n_jobs=-1) 之前工作正常，但突然出错.

来自 sklearn 导入数据集从 sklearn.model_selection 导入 cross_validate，StratifiedKFold从 sklearn.linear_model 导入 RidgeClassifier虹膜 = datasets.load_iris()iris_X = iris.datairis_y = iris.targetskf = StratifiedKFold(n_splits=5, random_state=0, shuffle=True)评分 = {'准确度':'准确度'}model_ridge = RidgeClassifier(random_state=0)分数 = cross_validate(estimator=model_ridge,X=虹膜_X，y=iris_y,得分=得分，cv=skf.split(iris_X, iris_y),n_jobs=-1)

<块引用>

线程 QueueManagerThread 中的异常:

回溯(最近一次调用最后一次):文件C:\ProgramData\Anaconda3\envs\py36\lib\threading.py"，第 916 行，在 _bootstrap_innerself.run()

文件C:\ProgramData\Anaconda3\envs\py36\lib\threading.py"，第 864 行，运行中self._target(*self._args, **self._kwargs)

文件C:\ProgramData\Anaconda3\envs\py36\lib\site-packages\sklearn\externals\joblib\externals\loky\process_executor.py"，第615行，在_queue_management_worker准备 = 等待(读者 + worker_sentinels)

文件C:\ProgramData\Anaconda3\envs\py36\lib\multiprocessing\connection.py"，第859行，等待ready_handles = _exhaustive_wait(waithandle_to_obj.keys(), timeout)

文件C:\ProgramData\Anaconda3\envs\py36\lib\multiprocessing\connection.py"，第 791 行，在 _exhaustive_waitres = _winapi.WaitForMultipleObjects(L, False, timeout)

ValueError: 最多需要 63 个句柄，得到一个长度为 65 的序列

解决方案

听起来您在那里运行了很多内核！设置 n_jobs=60 这应该可以工作.在此处查看更多详细信息 https://github.com/psf/black/issues/564

When I set n_jobs=-1 I get error and if I set n_jobs equal big value (n_jobs=100), but if set smaller value (e.g. n_jobs=32), it works fine. I've tried reinstall scikit-learn and joblib packages, but to no avail. Also, it (n_jobs=-1) works fine previously, but suddenly go wrong.

from sklearn import datasets
from sklearn.model_selection import cross_validate, StratifiedKFold
from sklearn.linear_model import RidgeClassifier

iris = datasets.load_iris()
iris_X = iris.data
iris_y = iris.target

skf = StratifiedKFold(n_splits=5, random_state=0, shuffle=True)
scoring = {'accuracy': 'accuracy'}

model_ridge = RidgeClassifier(random_state=0)
scores = cross_validate(estimator=model_ridge, 
                            X=iris_X,
                            y=iris_y,
                            scoring=scoring,
                            cv=skf.split(iris_X, iris_y),
                            n_jobs=-1)

Exception in thread QueueManagerThread:

Traceback (most recent call last): File "C:\ProgramData\Anaconda3\envs\py36\lib\threading.py", line 916, in _bootstrap_inner self.run()

File "C:\ProgramData\Anaconda3\envs\py36\lib\threading.py", line 864, in run self._target(*self._args, **self._kwargs)

File "C:\ProgramData\Anaconda3\envs\py36\lib\site-packages\sklearn\externals\joblib\externals\loky\process_executor.py", line 615, in _queue_management_worker ready = wait(readers + worker_sentinels)

File "C:\ProgramData\Anaconda3\envs\py36\lib\multiprocessing\connection.py", line 859, in wait ready_handles = _exhaustive_wait(waithandle_to_obj.keys(), timeout)

File "C:\ProgramData\Anaconda3\envs\py36\lib\multiprocessing\connection.py", line 791, in _exhaustive_wait res = _winapi.WaitForMultipleObjects(L, False, timeout)

ValueError: need at most 63 handles, got a sequence of length 65

解决方案

Sounds like you're running a lot of cores there! Set n_jobs=60 and this should work. See more details here https://github.com/psf/black/issues/564

这篇关于线程 QueueManagerThread 中的异常 - scikit-learn的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！