并行组合进度条和过程 [英] do parallel combine progress bar and process

问题描述

在将要并行运行的过程与进度条的创建结合在一起时遇到问题.

I'm having issues to combine the process that I want to run in parallel and the creation of the progress bar.

我对该过程的代码是:

pred_pnn <- function(x, nn){
  xlst <- split(x, 1:nrow(x))
  pred <- foreach(i = xlst,.packages = c('tcltk', 'foreach'), .combine = rbind) 
  %dopar% 
{ mypb <- tkProgressBar(title = "R progress bar", label = "",
                        min = 0, max = max(jSeq), initial = 0, width = 300)
  foreach(j = jSeq) %do% {Sys.sleep(.1)
  setTkProgressBar(mypb, j, title = "pb", label = NULL)
  }  
  library(pnn)
 data.frame(prob = guess(nn, as.matrix(i))$probabilities[1], row.names = NULL)
}
}

我将我的代码与此处一起使用的代码合并了

I combined my code and the one that comes form here

但是没有编译.我收到语法错误，但找不到.

but didn't compile. I get a syntax error, but I can't find it.

我尝试了其他代码:

pred_pnn <- function(x, nn){
  xlst <- split(x, 1:nrow(x))
  pred <- foreach(i = xlst, .combine = rbind) %dopar% 
{library(pnn)
 cat(i, '\n')
 data.frame(prob = guess(nn, as.matrix(i))$probabilities[1], row.names = NULL)
}
}

但我也收到错误消息.

推荐答案

在某些情况下，您尝试使用的方法可能行得通，但这并不是一个好的通用解决方案.我想做的是在主进程中(在foreach循环之外)创建一个进度条，然后让foreach在返回任务时更新该进度条.不幸的是，没有一个后端支持这一点.可以使用Combine函数技巧来做到这一点，但前提是您使用的是支持即时调用Combine函数的后端，而doParallel，doSNOW和doMC则不支持.这些后端不会立即调用组合，因为它们是使用clusterApplyLB和mclapply之类的函数实现的，这些函数不支持钩子，以使用户提供的代码在返回任务时得以执行.

The approach that you're trying to use might work under certain circumstances, but it isn't a good general solution. What I would want to do is to create a progress bar in the master process (outside of the foreach loop) and then have foreach update that progress bar as tasks are returned. Unfortunately, none of the backends support that. It's possible to do that using combine function tricks, but only if you're using a backend that supports calling the combine function on-the-fly, which doParallel, doSNOW and doMC do not. Those backends don't call combine on the fly because they are implemented using functions such as clusterApplyLB and mclapply which don't support a hook to allow user supplied code to be executed when tasks are returned.

因为我看到了对foreach中进度条支持的兴趣，所以我修改了doSNOW包以添加对doSNOW特定的"progress"选项的支持，然后将代码检查到R-Forge网站中.它利用了snow程序包中的一些较低级别的功能，不幸的是，这些功能并未由parallel程序包导出.

Because I've seen interest in progress bar support in foreach, I modified the doSNOW package to add support for a doSNOW-specific "progress" option, and I checked the code into the R-Forge website. It makes use of some lower level functions in the snow package which unfortunately are not exported by the parallel package.

如果要尝试此新功能，则需要从R-Forge安装doSNOW.我是使用以下命令在MacBook Pro上完成此操作的:

If you want to try out this new feature, you will need to install doSNOW from R-Forge. I did this on my MacBook Pro using the command:

install.packages("doSNOW", repos="http://R-Forge.R-project.org", type="source")

这是一个简单的示例脚本，演示了实验性的"progess"选项:

Here is a simple example script that demonstrates the experimental "progess" option:

library(doSNOW)
library(tcltk)
cl <- makeSOCKcluster(3)
registerDoSNOW(cl)
pb <- tkProgressBar(max=100)
progress <- function(n) setTkProgressBar(pb, n)
opts <- list(progress=progress)
r <- foreach(i=1:100, .options.snow=opts) %dopar% {
  Sys.sleep(1)
  sqrt(i)
}

更新

progress选项现在在CRAN的最新版本的doSNOW中可用.

The progress option is now available in the latest version of doSNOW on CRAN.

这篇关于并行组合进度条和过程的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！