使用更少的内存将行追加到numpy数组 [英] Appending rows to numpy array using less memory

问题描述

我有以下问题.我需要通过添加行和列来更改一个Numpy数组的形状以匹配另一个Numpy数组的形状.

I have the following problem. I need to change the shape of one Numpy array to match the shape of another Numpy array by adding rows and columns.

让我们说这是需要更改的数组:

Let's say this is the array that needs to be changed:

change_array = np.random.rand(150, 120)

这是引用数组:

reference_array = np.random.rand(200, 170)

要匹配形状，我要使用以下功能添加包含零的行和列:

To match the shapes I'm adding rows and columns containing zeros, using the following function:

def match_arrays(change_array, reference_array):

    cols = np.zeros((change_array.shape[0], (reference_array.shape[1] - change_array.shape[1])), dtype=np.int8)
    change_array = np.append(change_array, cols, axis=1)
    rows = np.zeros(((reference_array.shape[0] - change_array.shape[0]), reference_array.shape[1]), dtype=np.int8)
    change_array = np.append(change_array, rows, axis=0)
    return change_array

可以完美地工作，并将change_array的形状更改为reference_array的形状.但是，使用此方法，需要在内存中将阵列复制两次.我了解Numpy如何需要在内存中制作数组的副本，以便有空间附加行和列.

Which perfectly works and changes the shape of change_array to the shape of reference_array. However, using this method, the array needs to be copied twice in memory. I understand how Numpy needs to make a copy of the array in memory in order to have space to append the rows and columns.

由于我的数组可能变得非常大，因此我正在寻找另一种内存效率更高的方法，该方法可以实现相同的结果.谢谢！

As my arrays can get very large I am looking for another, more memory efficient method, that can achieve the same result. Thanks!

推荐答案

以下是几种方法.在代码示例中，我将使用以下数组:

Here are a couple ways. In the code examples, I'll use the following arrays:

In [190]: a
Out[190]: 
array([[12, 11, 15],
       [16, 15, 10],
       [16, 12, 13],
       [11, 19, 10],
       [12, 12, 11]])

In [191]: b
Out[191]: 
array([[70, 82, 83, 93, 97, 55],
       [50, 86, 53, 75, 75, 69],
       [60, 50, 76, 52, 72, 88],
       [72, 79, 66, 93, 58, 58],
       [57, 92, 71, 97, 91, 50],
       [60, 77, 67, 91, 91, 63],
       [60, 90, 91, 50, 86, 71]])

使用 numpy.pad :

In [192]: np.pad(a, [(0, b.shape[0] - a.shape[0]), (0, b.shape[1] - a.shape[1])], 'constant')
Out[192]: 
array([[12, 11, 15,  0,  0,  0],
       [16, 15, 10,  0,  0,  0],
       [16, 12, 13,  0,  0,  0],
       [11, 19, 10,  0,  0,  0],
       [12, 12, 11,  0,  0,  0],
       [ 0,  0,  0,  0,  0,  0],
       [ 0,  0,  0,  0,  0,  0]])

或者，使用功能更高效的版本，在该函数中，结果将预先分配为零数组，形状与 reference_array 相同，然后再分配 change_array 复制到结果中:

Or, use a more efficient version of your function, in which the result is preallocated as an array of zeros with the same shape as reference_array, and then the values in change_array are copied into the result:

In [193]: def match_arrays(change_array, reference_array):
     ...:     result = np.zeros(reference_array.shape, dtype=change_array.dtype)
     ...:     nrows, ncols = change_array.shape
     ...:     result[:nrows, :ncols] = change_array
     ...:     return result
     ...: 

In [194]: match_arrays(a, b)
Out[194]: 
array([[12, 11, 15,  0,  0,  0],
       [16, 15, 10,  0,  0,  0],
       [16, 12, 13,  0,  0,  0],
       [11, 19, 10,  0,  0,  0],
       [12, 12, 11,  0,  0,  0],
       [ 0,  0,  0,  0,  0,  0],
       [ 0,  0,  0,  0,  0,  0]])

这篇关于使用更少的内存将行追加到numpy数组的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！