将大xlsx文件导入R？ [英] Importing a big xlsx file into R?

问题描述

我想知道有没有人知道从大xlsx文件（〜20Mb）导入数据的方式。我试图使用xlsx和XLConnect库。不幸的是，两者都使用rJava和我总是获得相同的错误：

I'm wondering if anyone knows of a way to import data from a "big" xlsx file (~20Mb). I tried to use xlsx and XLConnect libraries. Unfortunately, both use rJava and I always obtain the same error:

> library(XLConnect)
> wb <- loadWorkbook("MyBigFile.xlsx")
Error: OutOfMemoryError (Java): Java heap space

或

> library(xlsx)
> mydata <- read.xlsx2(file="MyBigFile.xlsx")
Error in .jcall("RJavaTools", "Ljava/lang/Object;", "invokeMethod", cl,  : 
   java.lang.OutOfMemoryError: Java heap space

我还尝试在加载rJava之前修改java.parameters：

I also tried to modify the java.parameters before loading rJava:

> options( java.parameters = "-Xmx2500m")
> library(xlsx) # load rJava
> mydata <- read.xlsx2(file="MyBigFile.xlsx")
Error in .jcall("RJavaTools", "Ljava/lang/Object;", "invokeMethod", cl,  : 
   java.lang.OutOfMemoryError: Java heap space

或加载rJava后（这是有点笨，我想）：

or after loading rJava (this is a bit stupid, I think):

> library(xlsx) # load rJava
> options( java.parameters = "-Xmx2500m")
> mydata <- read.xlsx2(file="MyBigFile.xlsx")
Error in .jcall("RJavaTools", "Ljava/lang/Object;", "invokeMethod", cl,  : 
   java.lang.OutOfMemoryError: Java heap space

但没有什么可行的。有人有想法吗？

But nothing works. Does anyone have an idea?

推荐答案

当有人给我（另一个）Excel文件进​​行分析时，我偶然发现了这个问题。这一个甚至没有那么大，但无论什么原因，我遇到类似的错误：

I stumbled on this question when someone sent me (yet another) Excel file to analyze. This one isn't even that big but for whatever reason I was running into a similar error:

java.lang.OutOfMemoryError: GC overhead limit exceeded

根据@Dirk Eddelbuettel在上一个答案中的评论，我安装了openxlsx包（< a href =http://cran.r-project.org/web/packages/openxlsx/> http://cran.r-project.org/web/packages/openxlsx/ ）。然后运行：

Based on @Dirk Eddelbuettel's comment in a previous answer I installed the openxlsx package (http://cran.r-project.org/web/packages/openxlsx/). and then ran:

library("openxlsx")
mydf <- read.xlsx("BigExcelFile.xlsx", sheet = 1, startRow = 2, colNames = TRUE)

这只是我在看对于。易于使用和邪恶快。这是我的新BFF。感谢您的提示@Dirk E！

It was just what I was looking for. Easy to use and wicked fast. It's my new BFF. Thanks for the tip @Dirk E!

BTW，我不想从Dirk E偷取这个答案，所以如果他发布答案，请接受，而不是我的！

BTW, I don't want to poach this answer from Dirk E, so if he posts an answer, please accept it rather than mine!

这篇关于将大xlsx文件导入R？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！