具有数组作为输出的并行`for`循环 [英] Parallel `for` loop with an array as output

问题描述

如何并行运行for循环(以便可以使用Windows机器上的所有处理器)，结果是3维数组?我现在运行的代码大约需要一个小时，并且类似于:

How can I run a for loop in parallel (so I can use all the processors on my windows machine) with the result being a 3 dimension array? The code I have now takes about an hour to run and is something like:

guad = array(NA,c(1680,170,15))
for (r in 1:15)
{
  name = paste("P:/......",r,".csv",sep="")
  pp = read.table(name,sep=",",header=T)
    #lots of stuff to calculate x (which is a matrix)
  guad[,,r]= x  #
}

我一直在研究相关问题，以为可以使用foreach，但是找不到将矩阵组合成数组的方法.

I have been looking at related questions and thought I could use foreach but I couldn't find a way to combine the matrices into an array.

我是并行编程的新手，因此非常感谢您的帮助！

I am new to parallel programming so any help will be very much appreciated!

推荐答案

您可以使用abind函数使用foreach进行此操作.这是一个使用doParallel包作为并行后端的示例，该后端相当可移植:

You could do that with foreach using the abind function. Here's an example using the doParallel package as the parallel backend which is fairly portable:

library(doParallel)
library(abind)
cl <- makePSOCKcluster(3)
registerDoParallel(cl)
acomb <- function(...) abind(..., along=3)
guad <- foreach(r=1:4, .combine='acomb', .multicombine=TRUE) %dopar% {
  x <- matrix(rnorm(16), 4)  # compute x somehow
  x  # return x as the task result
}

这使用称为acomb的组合函数，该函数使用abind包中的abind函数将群集工作程序生成的矩阵组合成3维数组.

This uses a combine function called acomb that uses the abind function from the abind package to combine the matrices generated by the cluster workers into a 3 dimensional array.

在这种情况下，还可以使用cbind合并结果，然后修改dim属性，以将结果矩阵转换为3维数组:

In this case, you can also combine the results using cbind and then modify the dim attribute afterwards to convert the resulting matrix into a 3 dimensional array:

guad <- foreach(r=1:4, .combine='cbind') %dopar% {
  x <- matrix(rnorm(16), 4)  # compute x somehow
  x  # return x as the task result
}
dim(guad) <- c(4,4,4)

使用abind很有用，因为它可以以多种方式组合矩阵和数组.另外，请注意，重置dim属性可能会导致矩阵重复，这对于大型数组可能是个问题.

The use of abind is useful since it can combine matrices and arrays in a variety of ways. Also, be aware that resetting the dim attribute may cause the matrix to be duplicated which could be a problem for large arrays.

请注意，使用stopCluster(cl)在脚本末尾关闭群集是个好主意.

Note that it's a good idea to shutdown the cluster at the end of the script using stopCluster(cl).

这篇关于具有数组作为输出的并行`for`循环的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！