在Theano中循环(或矢量化)可变长度矩阵 [英] Loop over (or vectorize) variable length matrices in Theano

问题描述

我有一个矩阵L的列表，其中每个项M是一个x*n矩阵(x是一个变量，n是一个常数).

I have a list of matrices L, where each item M is a x*n matrix (x is a variable, n is a constant).

我想像以下Python代码一样为L中的所有项计算M'*M的总和(M'是M的转置):

I want to compute the sum of M'*M for all items in L (M' is the transpose of M) as the following Python code does:

for M in L:
  res += np.dot(M.T, M)

实际上，我想在Theano(不支持可变长度的多维数组)中实现此功能，并且我不想将所有矩阵填充到相同的大小，因为这会浪费太多空间(某些矩阵可以很大).

Actually I want to implement this in Theano (which doesn't support variable length multidimensional arrays), and I don't want to pad all matrices to the same size because that will waste too much space (some of the matrices can be very large).

有更好的方法吗?

修改:

L在Theano编译之前是众所周知的.

L is known before the Theano compilation.

修改:

从@DanielRenshaw和@Divakar收到了两个很好的答案，在情感上很难选择一个来接受.

received two excellent answers from @DanielRenshaw and @Divakar, emotionally difficult to choose one to accept.

推荐答案

您可以沿第一个轴填充输入数组，这是所有x的加法运算.因此，我们最终会得到一个高大的(X,n)数组，其中X =x1+x2+x3+.....可以对其进行转置，其点积及其自身将成为形状(n,n)的所需输出.所有这些都是通过利用功能强大的点积的纯矢量化解决方案来实现的.因此，实现将是-

You can just pad the input arrays along the first axis that is all the x's add up. Thus, we would end up with a tall (X,n) array, where X =x1+x2+x3+..... This can be transposed and its dot product with its self would be the desired output of shape (n,n). All of this is achieved with a pure vectorized solution leveraging the powerful dot product. Thus, the implementation would be -

# Concatenate along axis=0
Lcat = np.concatenate(L,axis=0)

# Perform dot product of the transposed version with self
out = Lcat.T.dot(Lcat)

运行时测试并验证输出-

Runtime tests and verify output -

In [116]: def vectoized_approach(L):
     ...:   Lcat = np.concatenate(L,axis=0)
     ...:   return Lcat.T.dot(Lcat)
     ...: 
     ...: def original_app(L):
     ...:   n = L[0].shape[1]
     ...:   res = np.zeros((n,n))
     ...:   for M in L:
     ...:     res += np.dot(M.T, M)
     ...:   return res
     ...: 

In [117]: # Input
     ...: L = [np.random.rand(np.random.randint(1,9),5)for iter in range(1000)]

In [118]: np.allclose(vectoized_approach(L),original_app(L))
Out[118]: True

In [119]: %timeit original_app(L)
100 loops, best of 3: 3.84 ms per loop

In [120]: %timeit vectoized_approach(L)
1000 loops, best of 3: 632 µs per loop

这篇关于在Theano中循环(或矢量化)可变长度矩阵的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！