向量化大型NumPy乘法 [英] Vectorize large NumPy multiplication
问题描述
我对计算大型NumPy数组感兴趣.我有一个大数组A
,其中包含一堆数字.我想计算这些数字的不同组合的总和.数据的结构如下:
I am interested in calculating a large NumPy array. I have a large array A
which contains a bunch of numbers. I want to calculate the sum of different combinations of these numbers. The structure of the data is as follows:
A = np.random.uniform(0,1, (3743, 1388, 3))
Combinations = np.random.randint(0,3, (306,3))
Final_Product = np.array([ np.sum( A*cb, axis=2) for cb in Combinations])
我的问题是,是否有一种更优雅,更省内存的方法来计算?当涉及到3-D数组时,我发现使用np.dot()
令人沮丧.
My question is if there is a more elegant and memory efficient way to calculate this? I find it frustrating to work with np.dot()
when a 3-D array is involved.
如果有帮助,Final_Product
的形状理想情况下应为(3743,306,1388).当前Final_Product
的形状为(306、3743、1388),所以我可以重塑形状以到达那里.
If it helps, the shape of Final_Product
ideally should be (3743, 306, 1388). Currently Final_Product
is of the shape (306, 3743, 1388), so I can just reshape to get there.
推荐答案
np.einsum
的一种vectorized
方法一键完成,没有任何额外的内存开销-
np.dot()
won't give give you the desired output , unless you involve extra step(s) that would probably include reshaping
. Here's one vectorized
approach using np.einsum
to do it one shot without any extra memory overhead -
Final_Product = np.einsum('ijk,lk->lij',A,Combinations)
对于完整性,如前所述,这里是np.dot
和reshaping
-
For completeness, here's with np.dot
and reshaping
as discussed earlier -
M,N,R = A.shape
Final_Product = A.reshape(-1,R).dot(Combinations.T).T.reshape(-1,M,N)
运行时测试并验证输出-
Runtime tests and verify output -
In [138]: # Inputs ( smaller version of those listed in question )
...: A = np.random.uniform(0,1, (374, 138, 3))
...: Combinations = np.random.randint(0,3, (30,3))
...:
In [139]: %timeit np.array([ np.sum( A*cb, axis=2) for cb in Combinations])
1 loops, best of 3: 324 ms per loop
In [140]: %timeit np.einsum('ijk,lk->lij',A,Combinations)
10 loops, best of 3: 32 ms per loop
In [141]: M,N,R = A.shape
In [142]: %timeit A.reshape(-1,R).dot(Combinations.T).T.reshape(-1,M,N)
100 loops, best of 3: 15.6 ms per loop
In [143]: Final_Product =np.array([np.sum( A*cb, axis=2) for cb in Combinations])
...: Final_Product2 = np.einsum('ijk,lk->lij',A,Combinations)
...: M,N,R = A.shape
...: Final_Product3 = A.reshape(-1,R).dot(Combinations.T).T.reshape(-1,M,N)
...:
In [144]: print np.allclose(Final_Product,Final_Product2)
True
In [145]: print np.allclose(Final_Product,Final_Product3)
True
这篇关于向量化大型NumPy乘法的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!