在Windows上的parLapply中使用Rcpp函数 [英] Using Rcpp function in parLapply on Windows

问题描述

我正在使用Rcpp和Windows上的并行计算进行R代码优化.我在parLapply中调用Rcpp函数时遇到麻烦.以下是示例

I'm doing R code optimization with Rcpp and parallel computing on Windows. I have a trouble calling Rcpp function in parLapply. The example is following

Rcpp代码(test.cpp)

Rcpp code (test.cpp)

#include <Rcpp.h>
using namespace Rcpp;

// [[Rcpp::export]]
NumericVector payoff( double strike, NumericVector data) {
    return pmax(data - strike, 0);
}

R代码

library(parallel)
library(Rcpp)

sourceCpp("test.cpp")

strike_list <- as.list(seq(10, 100, by = 5))

data <- runif(10000) * 50

# One core version
strike_payoff <- lapply(strike_list, payoff, data)

# Multiple cores version
numWorkers <- detectCores()
cl <- makeCluster(numWorkers, type = "PSOCK")
clusterExport(cl = cl,varlist = "payoff")
strike_payoff <- parLapply(cl, strike_list, payoff, data)

并行版本错误

Error in checkForRemoteErrors(val) : 
  8 nodes produced errors; first error: NULL value passed as symbol address   

我知道这是Windows的问题，因为mclapply在Linux上运行良好，但是我没有Windows强大的Linux机器.

I know that this is a Windows issue, as mclapply works well on Linux, but I don't have as powerful Linux machine as with Windows.

有什么想法要解决吗?

推荐答案

您需要在每个生成的进程中运行sourceCpp()调用，否则请获取它们的代码.现在，主要流程具有该功能，而产生的工人则没有.

You need to run the sourceCpp() call in each spawned process, or else get them your code. Right now the main process has the function, the spawned workers do not.

最简单的方法是构建一个程序包，并由每个工作进程加载它.

Easiest way is by building a package and have it loaded by each worker process.

这篇关于在Windows上的parLapply中使用Rcpp函数的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！