在函数内使用 parLapply 和 clusterExport [英] Using parLapply and clusterExport inside a function

问题描述

我在这里问了一个相关的问题，回复效果很好:使用并行的 parLapply:无法访问并行代码中的变量

I asked a related question here and the response worked well: using parallel's parLapply: unable to access variables within parallel code

问题是当我尝试在函数内部使用答案时，它不起作用，因为我认为它必须使用 clusterExport 的默认环境.我已经阅读了小插图并查看了帮助文件，但我的知识基础非常有限.我使用 parLapply 的方式我希望它的行为类似于 lapply 但它似乎没有.

The problem is when I try to use the answer inside of the function it won't work as I think it has to the default environment of clusterExport. I've read the vignette and looked at the help file but am approaching this with a very limited knowledge base. The way I used parLapply I expected it to behave similar to lapply but it doesn't appear to.

这是我的尝试:

par.test <- function(text.var, gc.rate=10){ 
    ntv <- length(text.var)
    require(parallel)
    pos <-  function(i) {
        paste(sapply(strsplit(tolower(i), " "), nchar), collapse=" | ")
    }
    cl <- makeCluster(mc <- getOption("cl.cores", 4))
    clusterExport(cl=cl, varlist=c("text.var", "ntv", "gc.rate", "pos"))
    parLapply(cl, seq_len(ntv), function(i) {
            x <- pos(text.var[i])
            if (i%%gc.rate==0) gc()
            return(x)
        }
    )
}

par.test(rep("I like cake and ice cream so much!", 20))

#gives this error message
> par.test(rep("I like cake and ice cream so much!", 20))
Error in get(name, envir = envir) : object 'text.var' not found

推荐答案

默认情况下 clusterExport 在 .GlobalEnv 中查找在 中命名的要导出的对象变量列表.如果您的对象不在 .GlobalEnv 中，您必须告诉 clusterExport 它可以在哪个环境中找到这些对象.

By default clusterExport looks in the .GlobalEnv for objects to export that are named in varlist. If your objects are not in the .GlobalEnv, you must tell clusterExport in which environment it can find those objects.

您可以将 clusterExport 更改为以下内容(我没有测试，但您在评论中说有效)

You can change your clusterExport to the following (which I didn't test, but you said works in the comments)

clusterExport(cl=cl, varlist=c("text.var", "ntv", "gc.rate", "pos"), envir=environment())

这样，它将在函数的环境中查找要导出的对象.

This way, it will look in the function's environment for the objects to export.

这篇关于在函数内使用 parLapply 和 clusterExport的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！