如何修复OutOfMemoryError（Java）：超出了GC开销限制？ [英] How to fix OutOfMemoryError (Java): GC overhead limit exceeded in r?

问题描述

我必须在文件夹列表中读取一个文件并将数据保存在R中。
我使用以下代码作为测试数据，它的工作原理。当我使用代码的实际数据然后
我得到这个错误

错误：OutOfMemoryError（Java）：超出GC开销限制
调用从：顶级

这是我为我的测试数据所做的一切

  parent.folder< C：/ Users / sandesh / Desktop / test_R
 sub.folder<  -  list.dirs（parent.folder，recursive = TRUE）[ -  1] 
文件<  -  file.path子文件夹sandesh1.xlsx）
库（xlsx）
库（plyr）
 fun<  - 函数（文件）{
 df < -  read.xlsx （文件，sheetIndex = 1）
} 
 df.big<  -  ldply（文件，乐趣）
  

解决方案

这是rJava中的一个典型问题。它在XLConnect文档中得到回答，该文档也使用rJava以与xlsx库相同的方式连接到excel。我引用了此处：

$ b $这是因为XLConnect（xlsx相同）需要将整个数据对象复制到JVM，以便
将其写入文件，并且必须使用一个JVM初始化JVM要修改
这个数量，可以将参数传递给R的JVM，就像您可以通过
rJava的选项支持命令行Java进程一样：

  options（java.parameters = -  Xmx1024m）
  

但是，请注意，当初始化JVM时，每个R会话这些参数被精确评估一次
- 通常一旦您加载了使用Java支持的第一个包，那么您应该这个早于
可能。

正如上面提到的，在加载任何库之前，在脚本开始时运行options功能 如果你是通过Rstudio重新运行它，请确保您在运行脚本之前重新启动它。

另请注意，仍然不能确定即使这样可以根据大小您要解析的档案。

I have to read a file in a list of folders and save data in R. I use following code for my test data and it works. When I use the code for the actual data then I get this error
Error: OutOfMemoryError (Java): GC overhead limit exceeded Called from: top level

This is what I have done for my test data

parent.folder <- "C:/Users/sandesh/Desktop/test_R"
sub.folder <- list.dirs(parent.folder, recursive =TRUE)[-1]
file <- file.path(sub.folder, "sandesh1.xlsx")
library(xlsx)
library(plyr)
fun <- function(file) {
  df <- read.xlsx(file, sheetIndex=1)
}
df.big <- ldply(file, fun)

解决方案

This is a typical problem in rJava. It is answered in the XLConnect documentation which also uses rJava to connect to excel the same way as the xlsx library. I quote from here:

"This is caused by the fact that XLConnect (same for xlsx) needs to copy your entire data object over to the JVM in order to write it to a file and the JVM has to be initialized with a fixed upper limit on its memory size. To change this amount, you can pass parameters to the R’s JVM just like you can to a command line Java process via rJava’s options support:

options(java.parameters = "- Xmx1024m")

Note, however, that these parameters are evaluated exactly once per R session when the JVM is initialized - this is usually once you load the first package that uses Java support, so you should do this as early as possible."

As it is mentioned above run the options function at the beginning of your script before loading any libraries and if you are running it through Rstudio make sure you restart it before you run the script.

Also, please note that it is still not certain that even this will work depending on the size of the file you are trying to parse.

这篇关于如何修复OutOfMemoryError（Java）：超出了GC开销限制？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！