创建不同形状的数组的对象数组时，如何防止numpy广播 [英] How to keep numpy from broadcasting when creating an object array of different shaped arrays

问题描述

我尝试使用np.save将不同形状的数组的列表存储为dtype=object数组(我知道我可以腌制该列表，但我很好奇如何做到这一点). 如果我这样做:

I try to store a list of different shaped arrays as a dtype=object array using np.save (I'm aware I could just pickle the list but I'm really curious how to do this). If I do this:

import numpy as np
np.save('test.npy', [np.zeros((2, 2)), np.zeros((3,3))])

有效. 但这:

np.save('test.npy', [np.zeros((2, 2)), np.zeros((2,3))])

给我一​​个错误:

ValueError: could not broadcast input array from shape (2,2) into shape (2)

我想np.save首先将列表转换为数组，所以我尝试了:

I guess np.save converts the list into an array first, so I tried:

x=np.array([np.zeros((2, 2)), np.zeros((3,3))])
y=np.array([np.zeros((2, 2)), np.zeros((2,3))])

具有相同的效果(第一个有效，第二个无效. 结果x的行为符合预期:

Which has the same effect (first one works, second one doesn't. The resulting x behaves as expected:

>>> x.shape
(2,)
>>> x.dtype
dtype('O')
>>> x[0].shape
(2, 2)
>>> x[0].dtype
dtype('float64')

我还尝试强制使用'object'dtype:

I also tried to force the 'object' dtype:

np.array([np.zeros((2, 2)), np.zeros((2,3))], dtype=object)

没有成功.似乎numpy尝试将具有相同一维的数组广播到新数组中，并且意识到它们的形状不同为时已晚.奇怪的是，它似乎只在某一时刻起作用了-所以我真的很好奇差异是什么，以及如何正确地做到这一点.

Without success. It seems numpy tries to broadcast the array with equal first dimension into the new array and realizes too late that their shape is different. Oddly it seems to have worked at one point - so I'm really curious what the difference is, and how to do this properly.

我已经弄清楚了它曾经工作过的情况:唯一的区别似乎是列表中的numpy数组具有另一种数据类型. 它适用于dtype('<f8')，但不适用于dtype('float64')，我什至不知道有什么区别.

I figured out the case it worked before: The only difference seems to be that the numpy arrays in the list have another data type. It works with dtype('<f8'), but it doesn't with dtype('float64'), I'm not even sure what the difference is.

我发现了一种非常非Python的方式来解决我的问题，我在这里添加了它，也许有助于理解我想做什么:

EDIT 2: I found a very non-pythonic way to solve my issue, I add it here, maybe it helps to understand what I wanted to do:

array_list=np.array([np.zeros((2, 2)), np.zeros((2,3))])
save_array = np.empty((len(array_list),), dtype=object)
for idx, arr in enumerate(array_list):
    save_array[idx] = arr
np.save('test.npy', save_array)

推荐答案

np.save要做的第一件事是

arr = np.asanyarray(arr)

是的，它正在尝试将您的列表变成数组.

So yes it is trying to turn your list into an array.

从任意大小的数组或列表构造对象数组很棘手. np.array(...)尝试创建尽可能高的维度数组，甚至尝试连接输入(如果可能).最确定的方法是执行您的操作-制作empty数组并填充它.

Constructing an object array from arbitrary sized arrays or lists is tricky. np.array(...) tries to create as high a dimensional array as it can, even attempting to concatenate the inputs if possible. The surest way is to do what you did - make the empty array and fill it.

一种构造对象数组的紧凑方式:

A slightly more compact way of constructing the object array:

In [21]: alist = [np.zeros((2, 2)), np.zeros((2,3))]
In [22]: arr = np.empty(len(alist), dtype=object)
In [23]: arr[:] = alist
In [24]: arr
Out[24]: 
array([array([[ 0.,  0.],
       [ 0.,  0.]]),
       array([[ 0.,  0.,  0.],
       [ 0.,  0.,  0.]])], dtype=object)

这是3种情况:

形状匹配的数组合并为3d数组:

Arrays that match in shape, combine into a 3d array:

In [27]: np.array([np.zeros((2, 2)), np.zeros((2,2))])
Out[27]: 
array([[[ 0.,  0.],
        [ 0.,  0.]],

       [[ 0.,  0.],
        [ 0.,  0.]]])
In [28]: _.shape
Out[28]: (2, 2, 2)

在第一维上不匹配的数组-创建对象数组

Arrays that don't match on the first dimension - create object array

In [29]: np.array([np.zeros((2, 2)), np.zeros((3,2))])
Out[29]: 
array([array([[ 0.,  0.],
       [ 0.,  0.]]),
       array([[ 0.,  0.],
       [ 0.,  0.],
       [ 0.,  0.]])], dtype=object)
In [30]: _.shape
Out[30]: (2,)

尴尬的中间情况(甚至可能被描述为错误).第一个尺寸匹配，但第二个尺寸不匹配:

And awkward intermediate case (which may even be described as a bug). The first dimensions match, but the second ones don't):

In [31]: np.array([np.zeros((2, 2)), np.zeros((2,3))])
...
ValueError: could not broadcast input array from shape (2,2) into shape (2)
       [ 0.,  0.]])], dtype=object)

就好像它初始化了一个(2,2,2)数组，然后发现(2,3)不适合.而且当前的逻辑不允许它像以前的场景一样备份和创建对象数组.

It's as though it initialized a (2,2,2) array, and then found that the (2,3) wouldn't fit. And the current logic doesn't allow it to backup and create the object array as it did in the previous scenario.

如果要将两个(2,2)数组放入对象数组，则必须使用创建和填充逻辑.

If you wanted to put the two (2,2) arrays in object array you'd have to use the create and fill logic.

这篇关于创建不同形状的数组的对象数组时，如何防止numpy广播的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！