如何在 R 中显示并行计算中的代码进度? [英] How to show the progress of code in parallel computation in R?
问题描述
我现在正在处理一个大型数据集,有些函数可能需要几个小时来处理.我想知道如何通过进度条或数字(1,2,3,...,100)显示代码的进度.我想将结果存储为具有两列的数据框.这是一个例子.谢谢.
I am now dealing with a large dataset and some functions may take hours to process. I wonder how I can show the progress of the code through a progress bar or number(1,2,3,...,100). And I want to store the result as a data frame with two columns. Here is an example. Thanks.
require(foreach)
require(doParallel)
require(Kendall)
cores=detectCores()
cl <- makeCluster(cores-1)
registerDoParallel(cl)
mydata=matrix(rnorm(8000*500),ncol = 500)
result=as.data.frame(matrix(nrow = 8000,ncol = 2))
pb <- txtProgressBar(min = 1, max = 8000, style = 3)
foreach(i=1:8000,.packages = "Kendall",.combine = rbind) %dopar%
{
abc=MannKendall(mydata[i,])
result[i,1]=abc$tau
result[i,2]=abc$sl
setTxtProgressBar(pb, i)
}
close(pb)
stopCluster(cl)
然而,当我运行代码时,我没有看到任何进度条出现,结果也不对.有什么建议吗?谢谢.
However, when I run the code, I did not see any progress bar showing up and the result is not right. Is there any suggestion? Thanks.
推荐答案
doSNOW 包支持进度条,而 doParallel 不支持.这是在示例中放置进度条的一种方法:
The doSNOW package has support for progress bars, while doParallel does not. Here's a way to put a progress bar in your example:
require(doSNOW)
require(Kendall)
cores <- parallel::detectCores()
cl <- makeSOCKcluster(cores)
registerDoSNOW(cl)
mydata <- matrix(rnorm(8000*500), ncol=500)
pb <- txtProgressBar(min=1, max=8000, style=3)
progress <- function(n) setTxtProgressBar(pb, n)
opts <- list(progress=progress)
result <-
foreach(i=1:8000, .packages="Kendall", .options.snow=opts,
.combine='rbind') %dopar% {
abc <- MannKendall(mydata[i,])
data.frame(tau=abc$tau, sl=abc$sl)
}
close(pb)
stopCluster(cl)
这篇关于如何在 R 中显示并行计算中的代码进度?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!