R:如何将包提供的方法导出到 PSOCK 集群? [英] R: How can I export methods provided by a package to a PSOCK cluster?

问题描述

在 R 中，我有一个矩阵列表，并且想对列表中的矩阵应用汇总函数.矩阵代表社交网络，因此我需要应用 ergm 包提供的一些专门的汇总函数.这些汇总统计信息包含在汇总方法中.我可以编写一个函数作为此摘要方法的包装器，并使用 lapply 将该函数应用于矩阵列表.

In R, I have a list of matrices and would like to apply a summary function to the matrices in the list. The matrices represent social networks, therefore I need to apply some specialized summary functions provided by the ergm package. These summary statistics are contained in a summary method. I can write a function as a wrapper around this summary method and use lapply to apply the function to the list of matrices.

然而，当我尝试使用 parallel 包中的 parLapply 或 parSapply 来并行化它时，结果看起来很奇怪.当我导出 summary.statistics 函数时，我什至收到一条错误消息.

However, when I try to parallelize this by using parLapply or parSapply from the parallel package, the results look weird. And when I export the summary.statistics function, I even get an error message.

一定要导出ergm包提供的summary方法到cluster对象吗?如果是这样，如何?以下代码是一个自包含的示例.

Do I have to export the summary method that is provided by the ergm package to the cluster object? If so, how? The following code is a self-contained example.

library("ergm")
library("parallel")

# create list of matrices
m <- matrix(rbinom(900, 1, 0.1), nrow = 30)
l <- list(m, m, m, m, m)

# write wrapper function that computes results
fun <- function(mat) {
  s <- summary(mat ~ edges + dsp(1))
  return(s)
}

cl <- makePSOCKcluster(2)  # create cluster object

test1 <- sapply(l, fun)  # works!
test2 <- parSapply(cl, l, fun)  # problem: results look weird!

clusterExport(cl, varlist = "summary.statistics")
test3 <- parSapply(cl, l, fun)  # problem: says method is not applicable!

推荐答案

不要导出在包中定义的函数，您应该使用以下内容在 worker 中加载包:

Instead of exporting functions that are defined in packages, you should load the package in the workers using something like:

clusterEvalQ(cl, library("ergm"))

您应该始终加载工作函数所需的所有包，因为它们不会因为包已被主服务器加载而自动加载.

You should always load all of the packages needed by the worker function, since they aren't loaded automatically just because the package has been loaded by the master.

这篇关于R:如何将包提供的方法导出到 PSOCK 集群?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！