光栅包占用所有硬盘 [英] raster package taking all hard drive

问题描述

我正在处理一个时间序列的栅格(modis ndvi 图像)来计算该序列的平均值和标准偏差.每个年度系列由 23 个 ndvi.tif 图像组成，每个图像 508Mb，因此总共需要处理 11Gb.以下是一年的脚本.我必须重复这一点很多年.

I am processing a time series of rasters (modis ndvi imagery) to calculate average and st.deviation of the series. Each yearly series is composed of 23 ndvi.tif images, each of 508Mb, so total is a big 11Gb to process. Below is the script for one year. I have to repeat this for a number of years.

library(raster)
library("rgeos")
filesndvi <- list.files(, pattern="NDVI.tif",full.names=TRUE) 
filesetndvi10 <- stack(filesndvi)
names(filesetndvi10)
avgndvi10<-mean(filesetndvi10)
desviondvi10 <- filesetndvi10 - avgndvi10
sumdesvioc <-sum(desviondvi10^2)
varndvi10  <- sumdesvioc/nlayers(filesetndvi10)
sdndvi10  <- sqrt(varndvi10)
cvndvi10  <- sdndvi10/avgndvi10

问题:该过程在硬盘驱动器中累积写入，直到它已满.不知道进程写在HD的什么地方.清理我发现的 HD 的唯一方法是重新启动.试过 rm，没有用.尝试关闭 RStudio，没有用.我在 4Gb RAM Asus UX31 和 256Gb HD 上使用 R 3.0.2 和 RStudio 0.98.994 和 Ubuntu 14.04.非常欢迎在每年计算后清理 HD 而不重新启动的任何想法.谢谢

The problem: the process writes accumulatively in the hard drive until it's full. Don't know where in the HD the process writes. Only way to clean the HD I've found is reboot. Tried rm, didn't work. Tried closing RStudio, didn't work. I'm using R 3.0.2 with RStudio 0.98.994 with Ubuntu 14.04 on a 4Gb RAM Asus UX31 with a 256Gb HD. Any thoughts to clean the HD after the calculation for each year without rebooting will be much welcome. Thanks

推荐答案

还有两件事需要考虑.首先，通过组合 calc 或 overlay 函数中的步骤来减少中间文件(这里没有太多范围，但有一些)，这也可以加快计算速度，因为从磁盘读取和写入磁盘的次数会减少.其次，控制删除特定文件.在 calc 和 overlay 函数中，您可以提供文件名，以便您可以删除不再需要的文件.但您也可以明确删除临时文件.首先删除指向这些文件的对象当然是一种很好的做法.这是一个基于您的示例.

There are two other things to consider. First, make fewer intermediate files by combining steps in calc or overlay functions (not too much scope for that here, but there is some), This can also speed up computations as there will be less reading from and writing to disk. Second, take control of deleting specific files. In the calc and overlay functions you can provide filenames such that you can remove the files you no longer need. But you can also delete the temp files explicitly. It is of course good practice to first remove the objects that point to these files. Here is an example based on yours.

library(raster)
# example data
set.seed(0)
ndvi <- raster(nc=10, nr=10)
n1 <- setValues(ndvi, runif(100) * 2 - 1)
n2 <- setValues(ndvi, runif(100) * 2 - 1)
n3 <- setValues(ndvi, runif(100) * 2 - 1)
n4 <- setValues(ndvi, runif(100) * 2 - 1)
filesetndvi10 <- stack(n1, n2, n3, n4)

nl <- nlayers(filesetndvi10)
avgndvi10 <- mean(filesetndvi10)
desviondvi10_2 <- overlay(filesetndvi10, avgndvi10, fun=function(x, y) (x - y)^2 , filename='over_tmp.grd')
sdndvi10 <- calc(desviondvi10_2, fun=function(x) sqrt(sum(x) / nl), filename='calc_tmp.grd')
cvndvi10  <- overlay(xsdndvi10, avgndvi10, fun=function(x,y) x / y, filename='cvndvi10.grd', overwrite=TRUE)

f <- filename(avgndvi10)
rm(avgndvi10, desviondvi10_2, sdndvi10)
file.remove(c(f, extension(f, '.gri')))
file.remove(c('over_tmp.grd', 'over_tmp.gri', 'calc_tmp.grd', 'calc_tmp.gri'))

要找出临时文件写在哪里看

To find out where temp files are written to look at

rasterOptions()

或者将路径作为变量获取:

or to get the path as a variable do:

dirname(rasterTmpFile()) 

要设置它的路径，使用

rasterOptions(tmpdir='a path')

这篇关于光栅包占用所有硬盘的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！