scipy 大稀疏矩阵 [英] scipy large sparse matrix

问题描述

我正在尝试使用大型 10^5x10^5 稀疏矩阵，但似乎遇到了 scipy:

I'm trying to use large 10^5x10^5 sparse matrices but seem to be running up against scipy:

n = 10 ** 5
x = scipy.sparse.rand(n, n, .001)

得到

ValueError: Trying to generate a random sparse matrix such as the
    product of dimensions is greater than 2147483647 - this is not
    supported on this machine

有谁知道为什么存在限制以及我是否可以避免它?(仅供参考，我使用的是具有 4GB 内存和 enthought 分布的 macbook air)

Does anyone know why limit is there and if I can avoid it? (fyi, I'm using a macbook air with 4gb memory and the enthought distribution)

推荐答案

这是由 scipy.sparse.rand() 的实现方式造成的限制.您可以滚动自己的随机矩阵生成来规避此限制:

This is a limitation which results from the way scipy.sparse.rand() is implemented. You can roll your own random matrix generation to circumvent this limitation:

n = 10 ** 5
density = 1e-3
ij = numpy.random.randint(n, size=(2, n * n * density))
data = numpy.random.rand(n * n * density)
matrix = scipy.sparse.coo.coo_matrix((data, ij), (n, n))

这篇关于scipy 大稀疏矩阵的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！