“索引太多"R中的大矩阵向量长度问题 [英] "Too many indices" big matrix vector length issue in R

问题描述

您好，提前致谢.我在 Windows Server 上使用 Rx64 版本 3.1.2，并从我试图在线性规划问题中使用的包 bigmemory 生成一个文件支持的大矩阵.该矩阵为 7062 行 x 364520 列，总共 2574240240 个条目(整数).

Hello and thanks in advance. I'm using Rx64 version 3.1.2 on a Windows Server and have a file-backed big matrix generated from the package bigmemory that I'm trying to use in a linear programming problem. The matrix is 7062 rows by 364520 columns for a total of 2574240240 entries (of integers).

当我为线性程序运行该行时，出现以下错误:

When I run the line for the linear program, I get the following error:

Error in GetElements.bm(x, i, j) : 
  Too many indices (>2^31-1) for extraction.

从我读到的数字 2147483647 是 R 允许的最大条目数，即使在 64 位进程上也是如此.我在这里读到我可以使用实验版"的 R 来解决这个问题，但我希望有一个最近可用的解决方案.我有以下支持输出来确认我的 R 版本:

That number, 2147483647, from what I read is the maximum number of entries R allows for any object even on 64-bit processes. I've read here that I can use the 'experimental version' of R to get around this but I was hoping there's a recently available solution. I have supporting output below to confirm my R version:

> version
               _                           
platform       x86_64-w64-mingw32          
arch           x86_64                      
os             mingw32                     
system         x86_64, mingw32             
status                                     
major          3                           
minor          1.2                         
year           2014                        
month          10                          
day            31                          
svn rev        66913                       
language       R                           
version.string R version 3.1.2 (2014-10-31)
nickname       Pumpkin Helmet 

> Sys.getenv("R_ARCH")
[1] "/x64"

推荐答案

如果您还可以提供线性程序行"，那将是最好的，但我的评论太长，无法放在上面.

It would be best if you could also provide the 'line for the linear program' but my comment is too long to put above.

@James 是正确的，目前 32 位被硬编码到 bigmemory 中.在查看 github 上的源代码后，我看到作者确实删除了约束，您可以使用以下命令下载最新版本:

@James is correct that 32-bit is hard-coded in to bigmemory at the moment. After glancing at the source on github I see the author did remove the constraint and you can download that newest version with:

library(devtools)
install_github("kaneplusplus/bigmemory")

如果没有这个限制，我相信你仍然会遇到问题.如果我假设正确，您也在使用补充包 bigalgebra.有一种方法可以通过重新安装下载的 tar ball 并使用以下命令安装 bigalgebra 来使用 64 位:

Without that limitation I believe you will still have a problem. If I assume correctly, you are also using the complementary package bigalgebra. There is a way to use 64-bit with bigalgebra by reintstalling the downloaded tar ball and installing with:

REFBLAS=1 R CMD INSTALL bigalgebra_0.8.4.tar.gz

注意 - 目前可用于 64 位的功能非常有限，并且由于当前的变通方法，会比非 64 位版本慢.

NOTE - the functions available for 64-bit are very limited currently and will be slower than the non-64-bit versions because of the currently workarounds.

我已与原作者联系，我正在更新软件包以使用 RcppArmadillo，它将提供更清晰的界面并处理 64 位问题.这目前正在我的 github 帐户 上的包的分支中进行.

I have been in contact with the original author and I am in the process of updating the packages to utilize RcppArmadillo which will provide a cleaner interface and deal with the 64-bit problem. This is currently in progress at my fork of the package on my github account.

这篇关于“索引太多"R中的大矩阵向量长度问题的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！