R中的并行化:如何“获取"源代码.在每个节点上? [英] Parallelization in R: how to "source" on every node?

问题描述

我使用以下方法创建了并行工作器(全部在同一台计算机上运行):

I have created parallel workers (all running on the same machine) using:

MyCluster = makeCluster(8)

如何使这8个节点中的每一个源都成为我编写的R文件? 我试过了:

How can I make every of these 8 nodes source an R-file I wrote? I tried:

clusterCall(MyCluster, source, "myFile.R")
clusterCall(MyCluster, 'source("myFile.R")')

和几个类似的版本.但是没有一个有效. 您能帮我找到错误吗?

And several similar versions. But none worked. Can you please help me to find the mistake?

非常感谢您！

推荐答案

以下代码可满足您的目的:

The following code serves your purpose:

library(parallel)

cl <- makeCluster(4)
clusterCall(cl, function() { source("test.R") })

## do some parallel work

stopCluster(cl)

此外，您可以使用clusterEvalQ()来执行相同的操作:

Also you can use clusterEvalQ() to do the same thing:

library(parallel)

cl <- makeCluster(4)
clusterEvalQ(cl, source("test.R"))

## do some parallel work

stopCluster(cl)

但是，两种方法之间存在细微的差异. clusterCall()在每个节点上运行一个函数，而clusterEvalQ()在每个节点上计算一个表达式.如果要获取源文件的变量列表，则clusterCall()会更易于使用，因为clusterEvalQ(cl,expr)会将任何expr视为表达式，因此在此放置变量并不方便.

However, there is subtle difference between the two methods. clusterCall() runs a function on each node while clusterEvalQ() evaluates an expression on each node. If you have a variable list of files to source, clusterCall() will be easier to use since clusterEvalQ(cl,expr) will regard any expr as an expression so it's not convenient to put a variable there.

这篇关于R中的并行化:如何“获取"源代码.在每个节点上?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！