将多个 *.bil 气候数据合并到 *.csv [英] Merge multiple *.bil climate data into *.csv

问题描述

我有超过 7,000 个 *.bil 文件，我正在尝试将其合并为一个 *.csv 文件并导出.我能够使用 raster 和 as.data.frame 读取 *.bil 文件:

I have over 7,000 files *.bil files I'm trying to merge into one *.csv file and export it. I am able to read the *.bil files using raster and as.data.frame :

setwd("/.../Prism Weather Data All/")
filenames <- list.files(path = "/.../Prism Weather Data All/", pattern = ".bil")
r = raster("PRISM_ppt_stable_4kmM2_189501_bil.bil")
test <- as.data.frame(r, na.rm=TRUE)

这会设置工作目录并抓取所有带有 *.bil 的文件.但是我只对一个文件进行光栅化并将其设置为 as.data.frame 来验证它是正确的，这很完美.但我想弄清楚如何将所有 7000 个文件(文件名)合并为一个.

This sets the working directory and grabs all files with *.bil. But I only raster one file and set as.data.frame to verify it's correct, which works perfect. But I'm trying to figure out how to merge all 7000 files (filenames) into one.

对此的任何帮助将不胜感激.提前致谢.

Any help with this would be appreciated. Thanks in advance.

推荐答案

假设 7000 是一个实数而不是一个近似值，并且每个文件中的所有数据都具有相同的结构(相同的列数和行数):

Assuming 7000 is a real number an not an approximation, and that all the data in each file is identically structured (same number of columns and rows):

setwd("/.../Prism Weather Data All/")

nc<- ## put the number of columns of each file (assuming they're all the same)
nr<- ## put the number of rows of each file (assuming they're all the same)

filenames <- list.files(path = "/.../Prism Weather Data All/", pattern = ".bil")

# initialize what is likely to be a large object
final.df<-as.data.frame(matrix(NA,ncol=7000*nc,nrow=nr)) 
counter=1
# loop through the files
for (i in filenames){
    r = raster(i)
    test <- as.data.frame(r, na.rm=TRUE)
    final.df[,counter:counter+nc]<-test
    counter<-counter+nc+1
}

# write the csv
write.csv(final.df,"final-filename.csv")

请记住，您的机器必须有足够的内存来保存所有数据，因为 R 需要在内存中有对象.

Keep in mind that your machine has to have enough memory to keep all the data, as R needs to have objects in memory.

如果列数因文件而异，您可以通过调整循环内 final.df 赋值中的索引来调整它，并相应地增加 counter.

If the number of columns differs from file to file, you can adjust that by adjusting the indexes in the final.df assignment inside the loop, and increasing counter accordingly.

产生预期结果

我认为 for 循环是完成此类工作的唯一方法.事实上，7000 个文件是一个相当大的集合，所以期待花一些时间来观察它的迭代.

I think a for loop is about the only way to do this sort of job. And indeed 7000 files is quite a large set, so expect to spend some time seeing it iterate.

setwd("/.../Prism Weather Data All/")

nc<- ## put the number of columns you expect the data in the files to have
nr<- ## put roughly the number of rows times 12 (if you plan to read a year worth of data)
     ## PLUS some tolerance, so you'll end up with an object actually larger than needed

filenames <- list.files(path = "/.../Prism Weather Data All/", pattern = ".bil")

# initialize what is likely to be a large object
final.df<-as.data.frame(matrix(NA,ncol=c,nrow=nr)) 
counter=1
# loop through the files
for (i in filenames){
    r = raster(i)
    test <- as.data.frame(r, na.rm=TRUE)
    numrow2<-nrow(test)
    final.df[counter:counter+numrow2,]<-test
    counter<-counter+numrow2+1
}

final.df[counter-1:nrow(final.df),]<-NULL  ## remove empty rows

# write the csv
write.csv(final.df,"final-filename.csv")

希望有帮助.

这篇关于将多个 *.bil 气候数据合并到 *.csv的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！