并行运行raster :: stackApply()函数 [英] Running raster::stackApply() function in parallel

问题描述

我正在尝试并行化此例子.

I'm trying to parallelize this example.

我要在一年中的一周内汇总一堆栅格.这是系列的样子:

I have a bunch of rasters that I am trying to aggregate by week of the year. Here is what this looks like in series:

# create a raster stack from list of GeoTiffs
tifs <- list.files(path = "./inputData/", pattern = "\\.tif$", full.names = TRUE)
r <- stack(tifs)

# get the date from the names of the layers and extract the week
indices <- format(as.Date(names(r), format = "X%Y.%m.%d"), format = "%U")
indices <- as.numeric(indices)

# calculate weekly means
r_week <- stackApply(r, indices, function(x) mean(x, na.rm = TRUE))

这是我尝试使用snow和pbapply进行并行化的尝试.

This is my attempt at parallelization using snow and pbapply.

# aggregate rasters in parallel
no_cores <- parallel::detectCores() - 1 

tryCatch({
  cl <- snow::makeCluster(no_cores, "SOCK")
  snow::clusterEvalQ(cl, {
    require(pacman)
    p_load(dplyr
           ,rts
           ,raster
           ,stringr
           ,pbapply
           ,parallel)
  })
  parallel::clusterExport(cl = cl, varlist = list("r", "indices"))
  r_week <-  pbapply::pbsapply(r, indices, stackApply(r, indices, function(x) mean(x, na.rm = TRUE)), simplify = TRUE, USE.NAMES = TRUE, cl = cl)
  snow::stopCluster(cl)
}, error=function(e){
  snow::stopCluster(cl)
  return(e)
}, finally = {
  try(snow::stopCluster(cl), silent = T)
})

stackApply()方法不采用群集参数，因此我试图将其包装在pbsapply()中.这将返回以下错误:

The stackApply() method does not take a cluster argument, so I'm trying to wrap it in a pbsapply(). This returns the following error:

<simpleError in get(as.character(FUN), mode = "function", envir = envir): object 'indices' of mode 'function' was not found>

推荐答案

我想我找到了使用raster::clusterR()方法的解决方法.它没有提供进度条.很高兴看看是否有人知道如何使用snow和pbapply来做到这一点.

I think I found a workaround using the raster::clusterR() method. It doesn't provide a progress bar though. It would be great to see if someone knows how to do this with snow and pbapply.

tryCatch({
  system.time({
  no_cores <- parallel::detectCores() - 1
  raster::beginCluster(no_cores)
  myFun <- function(x, ...) {
    mean(!is.na(x))
  }
  r_week <- raster::clusterR(r, stackApply, args=list(indices = indices, fun = myFun, na.rm = TRUE))
  raster::endCluster()})
}, error = function(e) {
  raster::endCluster()
  return(e)
}, finally = {
  try(raster::endCluster())
})

这篇关于并行运行raster :: stackApply()函数的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！