错误:找不到对象“.doSnowGlobals"? [英] error: object '.doSnowGlobals' not found?

问题描述

我正在尝试在 4 个节点上并行化代码(type = "SOCK").这是我的代码.

I'm trying to parallelize a code on 4 nodes(type = "SOCK"). Here is my code.

library(itertools)
library(foreach)
library(doParallel)
library(parallel)

workers <- ip address of 4 nodes
cl = makePSOCKcluster(workers, master="ip address of master")
registerDoParallel(cl)

z <- read.csv("ProcessedData.csv", header=TRUE, as.is=TRUE)
z <- as.matrix(z)


system.time({
  chunks <- getDoParWorkers()
  b <- foreach (these = isplitIndices(nrow(z),
                                      chunks=chunks),
                .combine = c) %dopar% {
                  a <- rep(0, length(these))
                  for (i in 1:length(these)) {
                    a[i] <- mean(z[these[i],])
                  }
                  a
                }
})

我收到此错误:

4 个节点产生错误；第一个错误:对象.doSnowGlobals"不是找到了.

4 nodes produced errors; first error: object '.doSnowGlobals' not found.

如果我使用的是 doMC，即使用同一台机器的内核，则此代码运行良好.但是当我尝试使用其他计算机进行并行计算时，出现上述错误.当我将其更改为 registerDoSNOW 时，错误仍然存​​在.

This code runs fine if I'm using doMC i.e using the same machine's cores. But when I try to use other computers for parallel computing I get the above error. When I change it to registerDoSNOW the error persists.

snow 和 DoSNOW 在集群中工作吗?我可以使用雪在本地主机上创建节点，但不能在集群上创建节点.有人在外面用雪吗?

Does snow and DoSNOW work in a cluster? I could create nodes on the localhost using snow but not on the cluster. Anyone out there using snow?

推荐答案

如果任何 worker 无法加载 doParallel 包，您就会收到此错误.您可以通过将 doParallel 安装到某个目录并通过.libPaths"将 master 指向它来实现这一点:

You can get this error if any of the workers are unable to load the doParallel package. You can make that happen by installing doParallel into some directory and pointing the master to it via ".libPaths":

> .libPaths('~/R/lib.test')
> library(doParallel)
> cl <- makePSOCKcluster(3, outfile='')
starting worker pid=26240 on localhost:11566 at 13:47:59.470
starting worker pid=26248 on localhost:11566 at 13:47:59.667
starting worker pid=26256 on localhost:11566 at 13:47:59.864
> registerDoParallel(cl)
> foreach(i=1:10) %dopar% i
Warning: namespace ‘doParallel’ is not available and has been replaced
by .GlobalEnv when processing object ‘’
Warning: namespace ‘doParallel’ is not available and has been replaced
by .GlobalEnv when processing object ‘’
Warning: namespace ‘doParallel’ is not available and has been replaced
by .GlobalEnv when processing object ‘’
Error in checkForRemoteErrors(lapply(cl, recvResult)) : 
  3 nodes produced errors; first error: object '.doSnowGlobals' not found

当来自 doParallel 的函数在工作线程上反序列化时会发生警告.当函数执行并尝试访问在 doParallel 命名空间中定义的 .doSnowGlobal 时会发生错误，而不是在 .GlobalEnv 中.

The warning happens when a function from doParallel is deserialized on a worker. The error happens when the function is executed and tries to access .doSnowGlobal which is defined in the doParallel namespace, not in .GlobalEnv.

您还可以通过执行以下操作来验证 doParallel 在工作线程上是否可用:

You can also verify that doParallel is available on the workers by executing:

> clusterEvalQ(cl, library(doParallel))
Error in checkForRemoteErrors(lapply(cl, recvResult)) : 
  3 nodes produced errors; first error: there is no package called ‘doParallel’

这篇关于错误:找不到对象“.doSnowGlobals"?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！