r语言 - 并行运行 raster::stackApply() 函数

Question

我正在尝试并行化此示例。

我有一堆栅格，我正在尝试按一年中的一周进行聚合。这是系列中的样子：

# create a raster stack from list of GeoTiffs
tifs <- list.files(path = "./inputData/", pattern = "\.tif$", full.names = TRUE)
r <- stack(tifs)
# get the date from the names of the layers and extract the week
indices <- format(as.Date(names(r), format = "X%Y.%m.%d"), format = "%U")
indices <- as.numeric(indices)
# calculate weekly means
r_week <- stackApply(r, indices, function(x) mean(x, na.rm = TRUE))

这是我使用snow和pbapply进行并行化的尝试。

# aggregate rasters in parallel
no_cores <- parallel::detectCores() - 1 
tryCatch({
cl <- snow::makeCluster(no_cores, "SOCK")
snow::clusterEvalQ(cl, {
require(pacman)
p_load(dplyr
,rts
,raster
,stringr
,pbapply
,parallel)
})
parallel::clusterExport(cl = cl, varlist = list("r", "indices"))
r_week <-  pbapply::pbsapply(r, indices, stackApply(r, indices, function(x) mean(x, na.rm = TRUE)), simplify = TRUE, USE.NAMES = TRUE, cl = cl)
snow::stopCluster(cl)
}, error=function(e){
snow::stopCluster(cl)
return(e)
}, finally = {
try(snow::stopCluster(cl), silent = T)
})

stackApply()方法不接受集群参数，所以我试图将其包装在pbsapply()中。这将返回以下错误：

<simpleError in get(as.character(FUN), mode = "function", envir = envir): object 'indices' of mode 'function' was not found>

Answer 1

我想我找到了使用raster::clusterR()方法的解决方法。不过，它不提供进度条。很高兴看到是否有人知道如何用snow和pbapply做到这一点.

tryCatch({
system.time({
no_cores <- parallel::detectCores() - 1
raster::beginCluster(no_cores)
myFun <- function(x, ...) {
mean(!is.na(x))
}
r_week <- raster::clusterR(r, stackApply, args=list(indices = indices, fun = myFun, na.rm = TRUE))
raster::endCluster()})
}, error = function(e) {
raster::endCluster()
return(e)
}, finally = {
try(raster::endCluster())
})

Answer 2

尝试将progress='text'添加到stackApply参数中。它在非并行化版本中工作正常。祝你好运！