应用SVD会立即引发内存错误吗? [英] Applying SVD throws a Memory Error instantaneously?

问题描述

我正在尝试将SVD应用于经过一些文本处理(最终目的是执行潜在语义分析)后获得的矩阵(3241 x 12596)，但我无法理解为什么这是我的64位代码机器有16GB RAM.调用svd(self.A)的时刻，将引发错误.准确的错误如下:

I am trying to apply SVD on my matrix (3241 x 12596) that was obtained after some text processing (with the ultimate goal of performing Latent Semantic Analysis) and I am unable to understand why this is happening as my 64-bit machine has 16GB RAM. The moment svd(self.A) is called, it throws an error. The precise error is given below:

Traceback (most recent call last):
  File ".\SVD.py", line 985, in <module>
    _svd.calc()
  File ".\SVD.py", line 534, in calc
    self.U, self.S, self.Vt = svd(self.A)
  File "C:\Python26\lib\site-packages\scipy\linalg\decomp_svd.py", line 81, in svd
    overwrite_a = overwrite_a)
MemoryError

所以我尝试使用

self.U, self.S, self.Vt = svd(self.A, full_matrices= False)

这一次，它引发了以下错误:

and this time, it throws the following error:

Traceback (most recent call last):
  File ".\SVD.py", line 985, in <module>
    _svd.calc()
  File ".\SVD.py", line 534, in calc
    self.U, self.S, self.Vt = svd(self.A, full_matrices= False)
  File "C:\Python26\lib\site-packages\scipy\linalg\decomp_svd.py", line 71, in svd
    return numpy.linalg.svd(a, full_matrices=0, compute_uv=compute_uv)
  File "C:\Python26\lib\site-packages\numpy\linalg\linalg.py", line 1317, in svd
    work = zeros((lwork,), t)
MemoryError

这是否应该是Numpy无法处理的大型矩阵，并且在此阶段我可以做些什么而无需更改方法本身?

Is this supposed to be such a large matrix that Numpy cannot handle and is there something that I can do at this stage without changing the methodology itself?

推荐答案

显然，由于@Ferdinand Beyer的帮助，我没有注意到我在64位版本上使用的是32位版本的Python机器.

Apparently, as it turns out, thanks to @Ferdinand Beyer, I did not notice that I was using a 32-bit version of Python on my 64-bit machine.

使用64位版本的Python并重新安装所有库即可解决此问题.

Using a 64-bit version of Python and reinstalling all the libraries solved the problem.

这篇关于应用SVD会立即引发内存错误吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！