使用 ClusterEvalQ 编译 Rcpp 函数 [英] Compiling Rcpp functions using ClusterEvalQ

问题描述

我正在处理一个需要在 R 中进行并行处理的项目，而且我是 doparallel 包的新手.我想做的是使用并行化的 foreach 循环.由于问题的性质，这个 foreach 循环将需要执行多次.我遇到的问题是我在循环中使用了 cppfunction 和 cfunction.

I am working on a project that requires parallel processing in R, and I am new to the doparallel package. What I would like to do is use a parallelized foreach loop. Due to the nature of the problem, this foreach loop will need to be executed many times. The problem I am having is that I use cppfunction and cfunction within the loop.

目前的解决方法是为集群调用clusterEvalQ()并编译相关函数.但是，这非常慢(4 核约 10 秒).我在下面包含了相关代码.有没有办法加快这个速度?谢谢.

The current work around is to call clusterEvalQ() for the cluster and to compile the relevant functions. However, this is extremely slow (~10 seconds for 4 cores). I have included the relevant code below. Is there any way to speed this up? Thanks.

clusterEvalQ(cl, {
library("inline")
library("Rcpp")
source("C_functions.R")
}) 

推荐答案

是的，有一种方法可以加快编译速度，只需一次.

Yes, there is a way to speed it up by taking the compilation hit only once.

特别是，将所有已编译的代码移动到R包中.从那里，将 R 包安装到集群上，然后加载该包.在并行代码中，调用包中的函数.

In particular, move all the compiled code into an R package. From there, install the R package onto the cluster and, then, load the package. Inside the parallel code, call the function in the package.

这是必需的，因为导入到 R 中的 C++ 函数是特定于会话的.因此，每个会话都需要自己的编译.编译是昂贵"的部分.

This is required because C++ functions imported into R are session-specific. As a result, each session requires its own compilation. The compilation is the "expensive" part.

另外，不要 使用 inline 包.相反，您应该使用 Rcpp 属性.

这篇关于使用 ClusterEvalQ 编译 Rcpp 函数的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！