通过雪在并行代码中使用 Rcpp 来创建集群 [英] Using Rcpp within parallel code via snow to make a cluster

问题描述

我用 Rcpp 编写了一个函数，并用 inline 编译它.现在，我想在不同的内核上并行运行它，但是我遇到了一个奇怪的错误.这是一个最小的例子，函数funCPP1可以自己编译运行，但是不能被snow的clusterCall函数调用.该函数作为单个进程运行良好，但并行运行时出现以下错误:

I've written a function in Rcpp and compiled it with inline. Now, I want to run it in parallel on different cores, but I'm getting a strange error. Here's a minimal example, where the function funCPP1 can be compiled and runs well by itself, but cannot be called by snow's clusterCall function. The function runs well as a single process, but gives the following error when ran in parallel:

Error in checkForRemoteErrors(lapply(cl, recvResult)) : 
  2 nodes produced errors; first error: NULL value passed as symbol address

这是一些代码:

## Load and compile
library(inline)
library(Rcpp)
library(snow)
src1 <- '
     Rcpp::NumericMatrix xbem(xbe);
     int nrows = xbem.nrow();
     Rcpp::NumericVector gv(g);
     for (int i = 1; i < nrows; i++) {
      xbem(i,_) = xbem(i-1,_) * gv[0] + xbem(i,_);
     }
     return xbem;
'
funCPP1 <- cxxfunction(signature(xbe = "numeric", g="numeric"),body = src1, plugin="Rcpp")

## Single process
A <- matrix(rnorm(400), 20,20)
funCPP1(A, 0.5)

## Parallel
cl <- makeCluster(2, type = "SOCK") 
clusterExport(cl, 'funCPP1') 
clusterCall(cl, funCPP1, A, 0.5)

推荐答案

仔细想想——内联有什么作用?它为您创建一个 C/C++ 函数，然后将其编译并链接到一个可动态加载的共享库中.那个坐哪?在 R 的临时目录中.

Think it through -- what does inline do? It creates a C/C++ function for you, then compiles and links it into a dynamically-loadable shared library. Where does that one sit? In R's temp directory.

因此，您尝试了正确的做法，将调用该共享库的 R 前端 传送到另一个进程(该进程具有另一个临时目录！！)，但这并没有在那里获取 dll/so 文件.

So you tried the right thing by shipping the R frontend calling that shared library to the other process (which has another temp directory !!), but that does not get the dll / so file there.

因此建议是创建一个本地包，安装它并让两个雪进程加载和调用它.

Hence the advice is to create a local package, install it and have both snow processes load and call it.

(和往常一样:在 rcpp-devel 列表中可能有更优质的答案，比 SO 阅读的 Rcpp 贡献者更多.)

(And as always: better quality answers may be had on the rcpp-devel list which is read by more Rcpp constributors than SO is.)

这篇关于通过雪在并行代码中使用 Rcpp 来创建集群的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！