通过 doParallel 在并行集群上插入符号训练二进制 glm 失败 [英] caret train binary glm fails on parallel cluster via doParallel
问题描述
我已经看到有很多关于这个主题的问题，但似乎没有一个对我的问题给出令人满意的答案.我打算在 Windows 机器上将 caret::train() 与库 doParallel 结合使用.文档(插入符号包:9 Parallel Processing)告诉我它会如果找到已注册的集群，则默认并行运行(尽管它使用库 doMC).当我尝试使用 doParallel 设置集群并按照其文档 (doParallel 和 foreach 入门) 一切正常.当我注销集群并运行 caret::train() 时，一切正常.但是，当我创建一个新集群并尝试运行 caret::train() 时，它会产生错误 Error in serialize(data, node$con) : error writing to connection.我还包括下面的日志.我不明白 caret::train() 如何在非并行模式下工作，但在并行模式下却没有，尽管集群似乎已正确设置.
I have seen there are a lot of questions around this topic already but none seems to give a satisfying answer to my problem. I intend to use caret::train() in combination with library doParallel on a Windows machine. The documentation (The caret package: 9 Parallel Processing) tells me that it will run in parallel by default if it finds a registered cluster (although it uses library doMC). When I attempt setting up a cluster with doParallel and follow the example calculation in its documentation (Getting Started with doParallel and foreach) everything works fine. When I unregister the cluster and run caret::train() everything works fine. But when I create a new cluster and try running caret::train() it produces error Error in serialize(data, node$con) : error writing to connection. I also include the log below. I don't understand how caret::train() works in non-parallel mode but not in parallel mode it doesn't although the cluster seems to be correctly setup.
library(caret)
library(microbenchmark)
library(doParallel)
会话信息
sessionInfo()
R version 3.4.1 (2017-06-30)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 7 x64 (build 7601) Service Pack 1
Matrix products: default
locale:
[1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252
[3] LC_MONETARY=English_United States.1252 LC_NUMERIC=C
[5] LC_TIME=English_United States.1252
attached base packages:
[1] parallel stats graphics grDevices utils datasets methods base
other attached packages:
[1] doParallel_1.0.10 iterators_1.0.8 foreach_1.4.3 microbenchmark_1.4-2.1
[5] caret_6.0-76 ggplot2_2.2.1 lattice_0.20-35
loaded via a namespace (and not attached):
[1] Rcpp_0.12.11 compiler_3.4.1 nloptr_1.0.4 plyr_1.8.4 tools_3.4.1
[6] lme4_1.1-13 tibble_1.3.3 nlme_3.1-131 gtable_0.2.0 mgcv_1.8-17
[11] rlang_0.1.1 Matrix_1.2-10 SparseM_1.77 mvtnorm_1.0-6 stringr_1.2.0
[16] hms_0.3 MatrixModels_0.4-1 stats4_3.4.1 grid_3.4.1 nnet_7.3-12
[21] R6_2.2.2 survival_2.41-3 multcomp_1.4-6 TH.data_1.0-8 minqa_1.2.4
[26] readr_1.1.1 reshape2_1.4.2 car_2.1-5 magrittr_1.5 scales_0.4.1
[31] codetools_0.2-15 ModelMetrics_1.1.0 MASS_7.3-47 splines_3.4.1 pbkrtest_0.4-7
[36] colorspace_1.3-2 quantreg_5.33 sandwich_2.4-0 stringi_1.1.5 lazyeval_0.2.0
[41] munsell_0.4.3 zoo_1.8-0
运行来自 doParallel 文档的示例(无错误)
cores_2_use <- floor(0.8 * detectCores())
cl <- makeCluster(cores_2_use, outfile = "parallel_log1.txt")
registerDoParallel(cl)
x <- iris[which(iris[,5] != "setosa"), c(1,5)]
trials <- 100
temp <- microbenchmark(
r <- foreach(icount(trials), .combine=cbind) %dopar% {
ind <- sample(100, 100, replace=TRUE)
result1 <- glm(x[ind,2]~x[ind,1], family=binomial(logit))
coefficients(result1)}
)
parallel::stopCluster(cl)
foreach::registerDoSEQ()
模拟数据
x1 = rnorm(100) # some continuous variables
x2 = rnorm(100)
z = 1 + 2 * x1 + 3 * x2 # linear combination with a bias
pr = 1 / (1 + exp(-z)) # pass through an inv-logit function
y = rbinom(100, 1, pr) # bernoulli response variable
df = data.frame(y = as.factor(ifelse(y == 0, "no", "yes")), x1 = x1, x2 = x2)
运行 caret::train() 非并行(无错误)
# train control function
ctrl <-
trainControl(
method = "repeatedcv",
number = 10,
repeats = 5,
classProbs = TRUE,
summaryFunction = twoClassSummary)
# train function
microbenchmark(
glm_nopar =
train(y ~ .,
data = df,
method = "glm",
family = "binomial",
metric = "ROC",
trControl = ctrl),
times = 5)
#Unit: milliseconds
#expr min lq mean median uq max neval
#glm_nopar 691.9643 805.1762 977.1054 895.9903 1018.112 1474.284 5
运行 caret::train() 并行(错误)
cores_2_use <- floor(0.8 * detectCores())
cl <- makeCluster(cores_2_use, outfile = "parallel_log2.txt")
registerDoParallel(cl)
microbenchmark(
glm_par =
train(y ~ .,
data = df,
method = "glm",
family = "binomial",
metric = "ROC",
trControl = ctrl),
times = 5)
#Error in serialize(data, node$con) : error writing to connection
编辑(尝试不使用 parallel::makeCluster() 调用)
在 Linux 设置中(见下文)也尝试不使用 parallel::makeCluster() 调用，即如下所示，但会导致相同的错误.
EDIT (trying without parallel::makeCluster() call)
As in Linux setup (see below) also tried without parallel::makeCluster() call, i.e. as shown below but results in same error.
cores_2_use <- floor(0.8 * detectCores())
registerDoParallel(cores_2_use)
...
输出parallel_log1.txt
starting worker pid=3880 on localhost:11442 at 16:00:52.764
starting worker pid=3388 on localhost:11442 at 16:00:53.405
starting worker pid=9920 on localhost:11442 at 16:00:53.789
starting worker pid=4248 on localhost:11442 at 16:00:54.229
starting worker pid=3548 on localhost:11442 at 16:00:54.572
starting worker pid=5704 on localhost:11442 at 16:00:54.932
starting worker pid=7740 on localhost:11442 at 16:00:55.291
starting worker pid=2164 on localhost:11442 at 16:00:55.653
starting worker pid=7428 on localhost:11442 at 16:00:56.011
starting worker pid=6116 on localhost:11442 at 16:00:56.372
starting worker pid=1632 on localhost:11442 at 16:00:56.731
starting worker pid=9160 on localhost:11442 at 16:00:57.092
starting worker pid=2956 on localhost:11442 at 16:00:57.435
starting worker pid=7060 on localhost:11442 at 16:00:57.811
starting worker pid=7344 on localhost:11442 at 16:00:58.170
starting worker pid=6688 on localhost:11442 at 16:00:58.561
starting worker pid=9308 on localhost:11442 at 16:00:58.920
starting worker pid=9260 on localhost:11442 at 16:00:59.281
starting worker pid=6212 on localhost:11442 at 16:00:59.641
输出parallel_log2.txt
starting worker pid=17640 on localhost:11074 at 15:12:21.118
starting worker pid=7776 on localhost:11074 at 15:12:21.494
starting worker pid=15128 on localhost:11074 at 15:12:21.961
starting worker pid=13724 on localhost:11074 at 15:12:22.345
starting worker pid=17384 on localhost:11074 at 15:12:22.714
starting worker pid=8472 on localhost:11074 at 15:12:23.228
starting worker pid=8392 on localhost:11074 at 15:12:23.597
starting worker pid=17412 on localhost:11074 at 15:12:23.979
starting worker pid=15996 on localhost:11074 at 15:12:24.364
starting worker pid=16772 on localhost:11074 at 15:12:24.743
starting worker pid=18268 on localhost:11074 at 15:12:25.120
starting worker pid=13504 on localhost:11074 at 15:12:25.500
starting worker pid=5156 on localhost:11074 at 15:12:25.899
starting worker pid=13544 on localhost:11074 at 15:12:26.275
starting worker pid=1764 on localhost:11074 at 15:12:26.647
starting worker pid=8076 on localhost:11074 at 15:12:27.028
starting worker pid=13716 on localhost:11074 at 15:12:27.414
starting worker pid=14596 on localhost:11074 at 15:12:27.791
starting worker pid=15664 on localhost:11074 at 15:12:28.170
Loading required package: caret
Loading required package: lattice
Loading required package: ggplot2
loaded caret and set parent environment
starting worker pid=3932 on localhost:11442 at 16:01:44.384
starting worker pid=6848 on localhost:11442 at 16:01:44.731
starting worker pid=5400 on localhost:11442 at 16:01:45.098
starting worker pid=9832 on localhost:11442 at 16:01:45.475
starting worker pid=8448 on localhost:11442 at 16:01:45.928
starting worker pid=1284 on localhost:11442 at 16:01:46.289
starting worker pid=9892 on localhost:11442 at 16:01:46.632
starting worker pid=8312 on localhost:11442 at 16:01:46.991
starting worker pid=3696 on localhost:11442 at 16:01:47.349
starting worker pid=9108 on localhost:11442 at 16:01:47.708
starting worker pid=8548 on localhost:11442 at 16:01:48.083
starting worker pid=7288 on localhost:11442 at 16:01:48.442
starting worker pid=6872 on localhost:11442 at 16:01:48.801
starting worker pid=3760 on localhost:11442 at 16:01:49.145
starting worker pid=3468 on localhost:11442 at 16:01:49.503
starting worker pid=2500 on localhost:11442 at 16:01:49.862
starting worker pid=7200 on localhost:11442 at 16:01:50.205
starting worker pid=7820 on localhost:11442 at 16:01:50.564
starting worker pid=8852 on localhost:11442 at 16:01:50.923
Error in unserialize(node$con) :
ReadItem: unknown type 0, perhaps written by later version of R
Calls: <Anonymous> ... doTryCatch -> recvData -> recvData.SOCKnode -> unserialize
Execution halted
编辑(在 Ubuntu 上尝试)
图书馆
library(caret)
library(microbenchmark)
library(doMC)
sessionInfo()
R version 3.4.1 (2017-06-30)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.04.3 LTS
Matrix products: default
BLAS: /usr/lib/libblas/libblas.so.3.6.0
LAPACK: /usr/lib/lapack/liblapack.so.3.6.0
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=de_DE.UTF-8
[4] LC_COLLATE=en_US.UTF-8 LC_MONETARY=de_DE.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=de_DE.UTF-8 LC_NAME=C LC_ADDRESS=C
[10] LC_TELEPHONE=C LC_MEASUREMENT=de_DE.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] parallel stats graphics grDevices utils datasets methods base
other attached packages:
[1] doMC_1.3.4 iterators_1.0.8 foreach_1.4.3
[4] microbenchmark_1.4-2.1 caret_6.0-77 ggplot2_2.2.1
[7] lattice_0.20-35
loaded via a namespace (and not attached):
[1] Rcpp_0.12.11 ddalpha_1.2.1 compiler_3.4.1 DEoptimR_1.0-8
[5] gower_0.1.2 plyr_1.8.4 bindr_0.1 class_7.3-14
[9] tools_3.4.1 rpart_4.1-11 ipred_0.9-6 lubridate_1.6.0
[13] tibble_1.3.3 nlme_3.1-131 gtable_0.2.0 pkgconfig_2.0.1
[17] rlang_0.1.1 Matrix_1.2-11 RcppRoll_0.2.2 prodlim_1.6.1
[21] bindrcpp_0.2 withr_2.0.0 stringr_1.2.0 dplyr_0.7.1
[25] recipes_0.1.0 stats4_3.4.1 nnet_7.3-12 CVST_0.2-1
[29] grid_3.4.1 robustbase_0.92-7 glue_1.1.1 R6_2.2.2
[33] survival_2.41-3 lava_1.5 purrr_0.2.2.2 reshape2_1.4.2
[37] kernlab_0.9-25 magrittr_1.5 DRR_0.0.2 splines_3.4.1
[41] scales_0.4.1 codetools_0.2-15 ModelMetrics_1.1.0 MASS_7.3-47
[45] assertthat_0.2.0 dimRed_0.1.0 timeDate_3012.100 colorspace_1.3-2
[49] stringi_1.1.5 lazyeval_0.2.0 munsell_0.4.3
示例来自 doMC 和 foreach 入门
按预期工作.
example from Getting Started with doMC and foreach
Works as expected.
microbenchmark(
glm_nopar =
train(y ~ .,
data = df,
method = "glm",
family = "binomial",
metric = "ROC",
trControl = ctrl),
times = 5)
#Unit: seconds
# expr min lq mean median uq max neval
#glm_nopar 1.093237 1.098342 1.481444 1.102867 2.001443 2.111333 5
caret 与 Windows 等设置并行(给出错误)
cores_2_use <- floor(0.8 * parallel::detectCores())
cl <- parallel::makeCluster(cores_2_use, outfile = "parallel_log2_linux.txt")
registerDoMC(cl)
microbenchmark(
glm_par =
train(y ~ .,
data = df,
method = "glm",
family = "binomial",
metric = "ROC",
trControl = ctrl),
times = 5)
# Error in getOper(ctrl$allowParallel && getDoParWorkers() > 1) :(list) object cannot be coerced to type 'double'
parallel_log2_linux.txt
starting worker pid=6343 on localhost:11836 at 16:05:17.781
starting worker pid=6353 on localhost:11836 at 16:05:18.025
starting worker pid=6362 on localhost:11836 at 16:05:18.266
caret parallel 没有 parallel::makeCluster() 调用(无错误)
不清楚如何在此设置中定义日志输出.
caret parallel without parallel::makeCluster() call (no error)
Unclear how to define log output in this setup.
cores_2_use <- floor(0.8 * parallel::detectCores())
registerDoMC(cores_2_use)
microbenchmark(
glm_par =
train(y ~ .,
data = df,
method = "glm",
family = "binomial",
metric = "ROC",
trControl = ctrl),
times = 5)
#Unit: milliseconds
# expr min lq mean median uq max neval
# glm_par 991.8075 997.4397 1013.686 998.8241 1004.381 1075.978 5
推荐答案
我在另一台内核较少但代码设置相同的 Windows 10 机器上进行了尝试.但是，我使用了 Github 的 caret 开发版本(通过 devtools::install_github('topepo/caret/pkg/caret') 安装)以及 R 3.4.1并且无法重现该问题.并行集群运行没有问题，下面的代码.不幸的是，我无法访问原始的 Windows 7 工作站，以查看 caret 开发版本和/或更新的 R 版本是否仍然存在问题.
I tried on a different Windows 10 machine with fewer cores but equal code setup. However, I used development version of caret from Github (installed via devtools::install_github('topepo/caret/pkg/caret')) as well as R 3.4.1 and the issue could not be reproduced. The parallel cluster ran without issues with below code. Unfortunately, I don't have access to the original Windows 7 workstation to see if the issue persists with caret dev version and/or newer R version.
library(doParallel)
cores_2_use <- floor(0.8 * detectCores())
cl <- makeCluster(cores_2_use, outfile = "parallel_log.txt")
registerDoParallel(cl)
glm_par <-
microbenchmark(glm_par =
train(default ~ .,
data = benchmark_train_data,
method = "glm",
family = "binomial",
metric = "ROC",
trControl = ctrl),
times = 5
)
glm_par
#Unit: seconds
# expr min lq mean median uq max neval
# glm_par 13.14082 13.25298 16.77678 13.64924 13.78132 30.05955 5
编辑(非平行基准)
这是在一个内核上运行的相同代码(与上面有六个内核的并行相反) - 预计并行设置的性能会更好.
This is the same code running on one core (as opposed to parallel above with six cores) - would have expected an even better performance for the parallel setup.
#Unit: seconds
# expr min lq mean median uq max neval
# glm_nopar 25.44122 25.52031 25.64818 25.53692 25.56496 26.17751 5
这篇关于通过 doParallel 在并行集群上插入符号训练二进制 glm 失败的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
