反序列化错误(socklist[[n]]):从 Unix 上的连接读取错误 [英] Error in unserialize(socklist[[n]]) : error reading from connection on Unix

问题描述

我尝试在 20 个 CPU 的 Unix 机器上运行以下代码，使用 R foreach、parallel、doParallel 和 party 包(我的目标是让 party/varimp 功能在多个 CPU 上并行工作):

I have tried running the following code on a Unix machine with 20 CPU, using R foreach, parallel, doParallel, and party packages (my objective is to have the party / varimp function working on several CPUs in parallel):

parallel_compute_varimp <- function (object, mincriterion = 0, conditional = FALSE, threshold = 0.2, 
    nperm = 1, OOB = TRUE, pre1.0_0 = conditional) 
{
    response <- object@responses
    input <- object@data@get("input")
    xnames <- colnames(input)
    inp <- initVariableFrame(input, trafo = NULL)
    y <- object@responses@variables[[1]]
    error <- function(x, oob) mean((levels(y)[sapply(x, which.max)] != y)[oob])

    w <- object@initweights
    perror <- matrix(0, nrow = nperm * length(object@ensemble), ncol = length(xnames))
    colnames(perror) <- xnames

    data = foreach(b = 1:length(object@ensemble), .packages = c("party","stats"), .combine = rbind) %dopar%
    {
        try({
            tree <- object@ensemble[[b]]
            oob <- object@weights[[b]] == 0

            p <- .Call("R_predict", tree, inp, mincriterion, -1L, PACKAGE = "party")

            eoob <- error(p, oob)

            for (j in unique(varIDs(tree))) {
                for (per in 1:nperm) {
                    if (conditional || pre1.0_0) {
                      tmp <- inp
                      ccl <- create_cond_list(conditional, threshold, xnames[j], input)
                      if (is.null(ccl)) {
                        perm <- sample(which(oob))
                      }
                      else {
                        perm <- conditional_perm(ccl, xnames, input, tree, oob)
                      }
                      tmp@variables[[j]][which(oob)] <- tmp@variables[[j]][perm]
                      p <- .Call("R_predict", tree, tmp, mincriterion, -1L, PACKAGE = "party")
                    }
                    else {
                      p <- .Call("R_predict", tree, inp, mincriterion, as.integer(j), PACKAGE = "party")
                    }
                    perror[b, j] <- (error(p, oob) - eoob)
                }
            }

            ########
            # return data to the %dopar% loop data variable
            perror[b, ]
            ########

        }) # END OF TRY
    } # END OF LOOP WITH PARALLEL COMPUTING

    perror = data
    perror <- as.data.frame(perror)
    return(MeanDecreaseAccuracy = colMeans(perror))
}

environment(parallel_compute_varimp) <- asNamespace('party')


cl <- makeCluster(detectCores())
registerDoParallel(cl, cores = detectCores())
<...>
system.time(data.cforest.varimp <- parallel_compute_varimp(data.cforest, conditional = TRUE))

但我收到一个错误:

> system.time(data.cforest.varimp <- parallel_compute_varimp(data.cforest, conditional = TRUE))
Error in unserialize(socklist[[n]]) : error reading from connection
Timing stopped at: 58.302 13.197 709.307

代码在 4 个 CPU 上处理较小的数据集.

The code was working with a smaller dataset on 4 CPUs.

我的想法不多了.有人可以建议一种方法来实现我在并行 CPU 上运行派对包 varimp 功能的目标吗?

I am running out of ideas. Can someone suggest a way to reach my objective of running party package varimp function on parallel CPUs?

推荐答案

错误:

Error in unserialize(socklist[[n]]) : error reading from connection

表示主进程在调用反序列化以从套接字连接读取到其中一个工作进程时出错.这可能意味着相应的工人死亡，从而丢弃了套接字连接的末端.不幸的是，它可能因多种原因而消亡，其中许多原因非常特定于系统.

means that the master process got an error when calling unserialize to read from the socket connection to one of the workers. That probably means that the corresponding worker died, thus dropping its end of the socket connection. Unfortunately, it may have died for any number of reasons, many of which are very system specific.

您通常可以通过使用 makeCluster 的outfile"选项找出工作人员死亡的原因，这样工作人员生成的错误消息就不会被丢弃.我通常建议使用 outfile="" 如本答案中所述.请注意，outfile"选项在雪和并行包中的作用相同.

You can usually figure out why the worker died by using the makeCluster "outfile" option so that the error message generated by the worker isn't thrown away. I usually recommend using outfile="" as described in this answer. Note that the "outfile" option works the same in both the snow and parallel packages.

您还可以通过注册顺序后端来验证您的 foreach 循环在顺序执行时是否正常工作:

You could also verify that your foreach loop works correctly when executed sequentially by registering the sequential backend:

registerDoSEQ()

如果幸运的话，foreach 循环在顺序执行时会失败，因为通常更容易找出问题所在.

If you're lucky, the foreach loop will fail when executed sequentially, since it's usually easier to figure out what is going wrong.

这篇关于反序列化错误(socklist[[n]]):从 Unix 上的连接读取错误的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！