创建非常大的numpy数组时出现MemoryError [英] MemoryError when creating a very large numpy array

问题描述

我正在尝试创建一个非常大的numpy零数组，然后将值从另一个数组复制到该大型零数组中.我正在使用Pycharm，即使我尝试仅创建数组，也一直得到:MemoryError.这是我尝试创建零数组的方法:

I'm trying to create a very large numpy array of zeros and then copy values from another array into the large array of zeros. I am using Pycharm and I keep getting: MemoryError even when I try and only create the array. Here is how I've tried to create the array of zeros:

import numpy as np

last_array = np.zeros((211148,211148))

我已尝试按照以下问题将Pycharm中的内存堆从750m增加到1024m:

I've tried increasing the memory heap in Pycharm from 750m to 1024m as per this question: https://superuser.com/questions/919204/how-can-i-increase-the-memory-heap-in-pycharm, but that doesn't seem to help.

如果您需要进一步的澄清，请告诉我.谢谢！

Let me know if you'd like any further clarification. Thanks!

推荐答案

使用scipy中的稀疏数组功能:
scipy.sparse文档.

Look into using the sparse array capabilities within scipy:
scipy.sparse documentation.

此处有scipy.sparse库的一组示例和教程:
Scipy讲义:SciPy中的稀疏矩阵

There are a set of examples and tutorials on the scipy.sparse library here:
Scipy lecture notes: Sparse Matrices in SciPy

这可以帮助您解决内存问题，并使一切运行得更快.

This may help you solve your memory issues, as well as make everything run faster.

要创建一个空的稀疏数组，并按照您在注释中的要求在某些位置放置值:

To create an empty sparse array with values in certain positions as you asked in your comment:

有什么方法可以创建一个在某些位置具有值的空数组，例如:last_array [211147] [9]，但其他地方都是空的?

Is there any way to create an empty array with values in certain positions, such as: last_array[211147][9] but everywhere else would be empty?

from scipy.sparse import *
values = [42]
row_ind = [211147]
col_ind = [9] 
last_array = csc_matrix((values, (row_ind, col_ind)), shape=(211148,211148))

print(last_array[211147,9])

这篇关于创建非常大的numpy数组时出现MemoryError的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！