R foreach循环-程序包加载失败 [英] R foreach loop - package load fails

问题描述

我无法在foreach％dopar％循环中将任何程序包加载到并行进程.

I'm unable to load any packages to parallel processes in a foreach %dopar% loop.

我使用foreach和doSNOW软件包成功创建了4个SOCK集群，然后尝试运行一个琐碎的并行过程.它可以与％dopar％一起使用，而无需使用任何程序包，但是加载任意程序包会引发错误.

I successfully create 4 SOCK clusters using the foreach and doSNOW packages, then try to run a trivial parallel process. It works with %dopar% without using any packages, but loading an arbitrary package throws an error.

下面的示例代码，后跟会话信息，显示正在使用的软件包.

Sample code below, followed by session info showing the packages in use.

我以前没有出现过这种类型的代码.我公司最近的一些网络更改意味着我不得不更改某些设置(默认库路径等)，可能与该更改有关.不确定从哪里开始进行问题排查-非常感谢您提供的帮助！

I have used this type of code before without issues. Some recent network changes at my company meant I had to change some settings (default library path, etc), may be related to that change. Not sure where to start in troubleshooting what the issue is - any help greatly appreciated!

#load foreach and doSNOW packages, setup 4 clusters#

> require(foreach)

> require(doSNOW)

> registerDoSNOW(makeCluster(4, type = "SOCK"))

> getDoParWorkers()
[1] 4

> getDoParName()
[1] "doSNOW"

# %dopar% loop without loading any packages -- works OK #

> foreach(i=1:2) %dopar%

{
    i+1
}

[[1]]
[1] 2

[[2]]
[1] 3

# %dopar% loop with loading a package -- does not work #

> foreach(i=1:2,.packages="forecast") %dopar%

  {

      i+1

  }

Error in e$fun(obj, substitute(ex), parent.frame(), e$data) : 
  worker initialization failed: package or namespace load failed for        'forecast'

会议信息: R版本3.2.1(2015-06-18) 平台:x86_64-w64-mingw32/x64(64位) 在以下环境中运行:Windows 7 x64(内部版本7601)Service Pack 1

Session Info: R version 3.2.1 (2015-06-18) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 7 x64 (build 7601) Service Pack 1

语言环境: [1] LC_COLLATE =英语_美国.1252 LC_CTYPE =英语_美国.1252 LC_MONETARY =英语_美国.1252 LC_NUMERIC = C
[5] LC_TIME =英语_美国.1252

locale: [1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252 LC_MONETARY=English_United States.1252 LC_NUMERIC=C
[5] LC_TIME=English_United States.1252

附带的基本软件包: [1]统计图形grDevices utils数据集方法基础

attached base packages: [1] stats graphics grDevices utils datasets methods base

其他附件包: [1] WriteXLS_3.6.1 rlist_0.4 timeSeries_3012.99 reshape2_1.4.1 plyr_1.8.3 lubridate_1.3.3 lmtest_0.9-34点阵_0.20-31 knitr_1.10.5
[10] hts_4.5 Matrix_1.2-1 SparseM_1.6 ggplot2_1.0.1 data.table_1.9.4 car_2.0-25 Forecast_6.1 timeDate_3012.100 zoo_1.7-12
[19] doSNOW_1.0.12 snow_0.3-13 iterators_1.0.7 foreach_1.4.2

other attached packages: [1] WriteXLS_3.6.1 rlist_0.4 timeSeries_3012.99 reshape2_1.4.1 plyr_1.8.3 lubridate_1.3.3 lmtest_0.9-34 lattice_0.20-31 knitr_1.10.5
[10] hts_4.5 Matrix_1.2-1 SparseM_1.6 ggplot2_1.0.1 data.table_1.9.4 car_2.0-25 forecast_6.1 timeDate_3012.100 zoo_1.7-12
[19] doSNOW_1.0.12 snow_0.3-13 iterators_1.0.7 foreach_1.4.2

通过名称空间(未附加)加载: [1] Rcpp_0.11.6编译器_3.2.1 nloptr_1.0.4 tseries_0.10-34工具_3.2.1 lme4_1.1-8摘要_0.6.8备忘录_0.2.1 gtable_0.1.2 nlme_3.1-120
[11] mgcv_1.8-6 parallel_3.2.1 proto_0.3-10 stringr_1.0.0 grid_3.2.1 nnet_7.3-9 minqa_1.2.4 magrittr_1.5 scales_0.2.5 codetools_0.2-11 [21] MASS_7.3-40 splines_3.2.1 pbkrtest_0.4-2 colorspace_1.2-6 fracdiff_1.4-2quantreg_5.11 quadprog_1.5-5 stringi_0.5-5 munsell_0.4.2 chron_2.3-47

loaded via a namespace (and not attached): [1] Rcpp_0.11.6 compiler_3.2.1 nloptr_1.0.4 tseries_0.10-34 tools_3.2.1 lme4_1.1-8 digest_0.6.8 memoise_0.2.1 gtable_0.1.2 nlme_3.1-120
[11] mgcv_1.8-6 parallel_3.2.1 proto_0.3-10 stringr_1.0.0 grid_3.2.1 nnet_7.3-9 minqa_1.2.4 magrittr_1.5 scales_0.2.5 codetools_0.2-11 [21] MASS_7.3-40 splines_3.2.1 pbkrtest_0.4-2 colorspace_1.2-6 fracdiff_1.4-2 quantreg_5.11 quadprog_1.5-5 stringi_0.5-5 munsell_0.4.2 chron_2.3-47

可能有用的其他信息-.libPaths()位置以及makeCluster函数中的详细信息.

Other info that may be useful- .libPaths() location and the details from the makeCluster function.

> .libPaths()
 [1] "C:/Users/G082580/Documents/My Documents/R/R-3.2.1" "C:/R-3.2.1/library"                               

> makeCluster(4,type="SOCK",manual=TRUE)

使用以下命令在localhost上手动启动worker C:/R-32~1.1/bin/Rscript.exe"C:/Users/G082580/Documents/My Documents/R/R-3.2.1/snow/RSOCKnode.R" MASTER = localhost PORT = 11535 OUT =/dev/null SNOWLIB = C:/Users/G082580/Documents/My Documents/R/R-3.2.1

Manually start worker on localhost with C:/R-32~1.1/bin/Rscript.exe "C:/Users/G082580/Documents/My Documents/R/R-3.2.1/snow/RSOCKnode.R" MASTER=localhost PORT=11535 OUT=/dev/null SNOWLIB=C:/Users/G082580/Documents/My Documents/R/R-3.2.1

推荐答案

如果需要使用.libPaths函数在主服务器上加载程序包，则也需要在worker上调用它.本示例使用clusterCall函数将工作程序初始化为与主服务器相同:

If you need to use the .libPaths function in order to load packages on the master, you'll need to call it on the workers as well. This example uses the clusterCall function to initialize the workers to be the same as the master:

library(doSNOW)
cl <- makeSOCKcluster(4)
registerDoSNOW(cl)
clusterCall(cl, function(x) .libPaths(x), .libPaths())

这篇关于R foreach循环-程序包加载失败的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！