避免R循环并与雪并行化 [英] avoid R loop and parallelize with snow

问题描述

我有一个大循环，它将花费很长时间(〜100天).我希望通过snow库来加快它的速度，但是我对apply语句并不满意.这只是循环的一部分，但是如果我能弄清楚这部分，其余部分应该很简单.我可以使用一堆apply语句或循环，但是一个使用函数获取对象"p"的apply语句将是理想的.

I have a large loop that will take too long (~100 days). I'm hoping to speed it up with the snow library, but I'm not great with apply statements. This is only part of the loop, but if I can figure this part out, the rest should be straightforward. I'm ok with a bunch of apply statements or loops, but one apply statement using a function to get object 'p' would be ideal.

原始数据

dim(m1)   == x x    # x >>> 0
dim(m2)   == y x    # y >>> 0, y > x, y > x-10
dim(mout) == x x    
thresh    == x-10   #specific to my data, actual number probably unimportant
len(v1)   == y      #each element is a random integer, min==1, max==thresh 
len(v2)   == y      #each element is a random integer, min==1, max==thresh 

原始循环

p <- rep(NA,y)
for (k in 1:y){
    mout <- m1 * matrix(m2[k,],x,x)
    mout <- mout/sum(mout)

    if (v1[k] < thresh + 1){
        if(v2[k] < thresh + 1){
            p[k] <- out[v1[k],v2[k]]
        }
        if(v2[k] > thresh){
            p[k] <-  sum(mout[v1[k],(thresh+1):x])
        }
    }

    #do stuff with object 'p'
}

推荐答案

library(snow)
dostuff <- function(k){
    #contents of for-loop
    mout <- m1 * matrix(m2[k,],x,x)
    mout <- mout/sum(mout)

    if (v1[k] < thresh + 1){
        if(v2[k] < thresh + 1){
            p <- out[v1[k],v2[k]]
        }
        if(v2[k] > thresh){
            p <-  sum(mout[v1[k],(thresh+1):x])
        }
    }

    #etc etc

    return(list(p,
                other_vars))
}

exports = c('m1',
            'm2',
            'thresh',
            'v1',
            'x' ,
            'v2')
cl = makeSOCKcluster(4)
clusterExport(cl,exports)

loop <- as.array(1:y)
out <- parApply(cl,loop,1,dostuff)

p <- rep(NA,y)
for(k in 1:y){
    p[k]          <- out[[k]][[1]]
    other_vars[k] <- out[[k]][[2]]
}

这篇关于避免R循环并与雪并行化的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！