具有n_jobs = -1的sklearn Logistic回归实际上并未并行化 [英] sklearn Logistic Regression with n_jobs=-1 doesn't actually parallelize

问题描述

我正在尝试使用sklearn的逻辑回归训练庞大的数据集. 我已经设置了参数n_jobs = -1(也尝试过n_jobs = 5、10，...)，但是当我打开htop时，我看到它仍然只使用一个内核.

I'm trying to train a huge dataset with sklearn's logistic regression. I've set the parameter n_jobs=-1 (also have tried n_jobs = 5, 10, ...), but when I open htop, I can see that it still uses only one core.

这是否意味着逻辑回归仅忽略n_jobs参数?

Does it mean that logistic regression just ignores the n_jobs parameter?

我该如何解决?我真的需要这个过程来并行化...

How can I fix this? I really need this process to become parallelized...

P.S.我正在使用sklearn 0.17.1

P.S. I am using sklearn 0.17.1

推荐答案

并行进程后端也取决于求解器方法.如果要使用多核，则需要multiprocessing后端.

the parallel process backend also depends on the solver method. if you want to utilize multi core, the multiprocessing backend is needed.

但像'sag'这样的求解器只能使用threading后端.

but solver like 'sag' can only use threading backend.

并且在大多数情况下，它可能由于大量预处理而被阻止.

and also mostly, it can be blocked due to a lot of pre-processing.

这篇关于具有n_jobs = -1的sklearn Logistic回归实际上并未并行化的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！