R Windows 64 位版本中尚不支持长向量错误 [英] Long Vector Not Supported Yet Error in R Windows 64bit version

问题描述

我正在尝试测试当前 R 版本中的内存限制是什么.

I'm trying to test what the memory limitations in the current R version is.

runtest <- function(size) {
  x <- "testme"
  while(0<1) {
    x <- c(x, x)
    size <<- object.size(x)  # size of x when fail
  }
}

通过在笔记本电脑的控制台中运行 runtest(size)，我收到以下错误:

By running runtest(size) in the console on my laptop, I get the following error:

> runtest(size)
Error: cannot allocate vector of size 4.0 Gb
In addition: Warning messages:
1: In structure(.Call(C_objectSize, x), class = "object_size") :
  Reached total allocation of 7915Mb: see help(memory.size)
2: In structure(.Call(C_objectSize, x), class = "object_size") :
  Reached total allocation of 7915Mb: see help(memory.size)
3: In structure(.Call(C_objectSize, x), class = "object_size") :
  Reached total allocation of 7915Mb: see help(memory.size)
4: In structure(.Call(C_objectSize, x), class = "object_size") :
  Reached total allocation of 7915Mb: see help(memory.size)
> size
2147483736 bytes
> 

这个尺寸看起来非常接近人们之前提到的 2^31-1 的限制.因此，我尝试在我们升级后的具有 128GB RAM 的桌面上运行相同的代码，并将 64 位版本的快捷方式中的变量设置为 100GB 的最大内存使用量.这是我得到的新错误:

This size looks very close to the 2^31-1 limit that people have mentioned before. So then I tried running the same code on our upgraded desktop with 128GB of RAM and set the variable in the shortcut for the 64 bit version to the max memory usage of 100GB. This is the new error I get:

Error in structure(.Call(C_objectSize, ), class = "object_size"):
  long vectors not supported yet: unique.c: 1720
> size
8589934680 bytes
>

这个 8.5GB 的限制是否与在 Windows O/S(特别是 Windows 7 企业版)中运行有关?我认为 R 帮助文件 (http://stat.ethz.ch/R-manual/R-devel/library/base/html/Memory-limits.html) 解释了这一点，但我无法理解它在说什么(不是我的领域专业知识).

Does this 8.5GB limit have anything to do with running in Windows O/S (specifically Windows 7 Enterprise edition)? I think the R help file (http://stat.ethz.ch/R-manual/R-devel/library/base/html/Memory-limits.html) explains this, but I'm having trouble understanding what it's saying (not my area of expertise).

推荐答案

看源码size.c 和 unique.c 看起来用于改进 object.size 的散列还不支持长向量:

Looking at the source of size.c and unique.c it looks like the hashing used to improve object.size doesn't support long vectors yet:

/* Use hashing to improve object.size. Here we want equal CHARSXPs,
   not equal contents. */

和

/*  Currently the hash table is implemented as a (signed) integer
    array.  So there are two 31-bit restrictions, the length of the
    array and the values.  The values are initially NIL (-1).  O-based
    indices are inserted by isDuplicated, and invalidated by setting
    to NA_INTEGER.
*/

因此，令人窒息的是 object.size.不如调用numeric(2^36)看看能不能创建这么大的对象，(应该是64GB).

Therefore, it is object.size that is choking. How about calling numeric(2^36) to see if you can create a such a large object, (should be 64GB).

这篇关于R Windows 64 位版本中尚不支持长向量错误的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！