Theano 多张量作为输出 [英] Theano multiple tensors as output

问题描述

我正在使用 Theano 创建一个神经网络，但是当我尝试在一个列表中同时返回两个张量列表时出现错误:

I am using Theano to create a neural network, but when I try to return two lists of tensors at the same time in a list I get the error:

#This is the line that causes the error
#type(nabla_w) == <type 'list'>
#type(nabla_w[0]) == <class 'theano.tensor.var.TensorVariable'>
backpropagate = function(func_inputs, [nabla_w, nabla_b])

TypeError: Outputs must be theano Variable or Out instances. Received [dot.0, dot.0, dot.0, dot.0] of type <type 'list'>

我应该使用什么样的 Theano 结构将两个张量一起返回到一个数组中，以便我可以像这样检索它们:

What kind of Theano structure should I use to return the two tensors together in an array so I can retrieve them like this:

nabla_w, nabla_b = backpropagate(*args)

我尝试使用在基本张量功能页面 但这些都不起作用.(例如我尝试了堆栈或堆栈列表)

I tried using some of the things I found in the basic Tensor functionality page but none of those work. (For example I tried the stack or stacklists)

这是我在使用 theano.tensor.stack 或 stacklists 时遇到的错误:

Here is the error I get using theano.tensor.stack or stacklists:

ValueError: all the input array dimensions except for the concatenation axis must match exactly
Apply node that caused the error: Join(TensorConstant{0}, Rebroadcast{0}.0, Rebroadcast{0}.0, Rebroadcast{0}.0, Rebroadcast{0}.0)
Inputs shapes: [(), (1, 10, 50), (1, 50, 100), (1, 100, 200), (1, 200, 784)]
Inputs strides: [(), (4000, 400, 8), (40000, 800, 8), (160000, 1600, 8), (1254400, 6272, 8)]
Inputs types: [TensorType(int8, scalar), TensorType(float64, 3D), TensorType(float64, 3D), TensorType(float64, 3D), TensorType(float64, 3D)]
Use the Theano flag 'exception_verbosity=high' for a debugprint of this apply node.

代码的一些额外上下文:

A little extra context to the code:

weights = [T.dmatrix('w'+str(x)) for x in range(0, len(self.weights))]
biases = [T.dmatrix('b'+str(x)) for x in range(0, len(self.biases))]
nabla_b = []
nabla_w = []
# feedforward
x = T.dmatrix('x')
y = T.dmatrix('y')
activations = []
inputs = []
activations.append(x)
for i in xrange(0, self.num_layers-1):
    inputt = T.dot(weights[i], activations[i])+biases[i]
    activation = 1 / (1 + T.exp(-inputt))
    activations.append(activation)
    inputs.append(inputt)

delta = activations[-1]-y
nabla_b.append(delta)
nabla_w.append(T.dot(delta, T.transpose(inputs[-2])))

for l in xrange(2, self.num_layers):
    z = inputs[-l]
    spv = (1 / (1 + T.exp(-z))*(1 - (1 / (1 + T.exp(-z)))))
    delta = T.dot(T.transpose(weights[-l+1]), delta) * spv
    nabla_b.append(delta)
    nabla_w.append(T.dot(delta, T.transpose(activations[-l-1])))
T.set_subtensor(nabla_w[-l], T.dot(delta, T.transpose(inputs[-l-1])))
func_inputs = list(weights)
func_inputs.extend(biases)
func_inputs.append(x)
func_inputs.append(y)


backpropagate = function(func_inputs, [nabla_w, nabla_b])

推荐答案

Theano 不支持此操作.当你调用 theano.function(inputs,outputs) 时，输出只能是两件事:

This is not supported by Theano. When you call theano.function(inputs, outputs), outputs can be only 2 things:

1) 一个 Theano 变量2) Theano 变量列表

1) a Theano variable 2) a list of Theano variables

(2) 不允许您在顶级列表中有列表，因此您应该将输出中的列表展平.这将返回超过 2 个输出.

(2) does not allow you to have a list in the top level list, so you should flatten the lists in the outputs. This will return more then 2 outputs.

您的问题的一个可行解决方案是将内部列表复制到 1 个变量中.

A compossible solution to your problem is to have the inner list copied into 1 variable.

tensor_nabla_w = theano.tensor.stack(*nabla_w).

这要求 nabla_w 中的所有元素都是相同的形状.这将在计算图中添加一个额外的副本(所以它可能会慢一点).

This asks that all elements in nabla_w is are the same shape. This will add an extra copy in the computation graph (so it could be a little slower).

更新 1:修复对 stack() 的调用

Update 1: fix call to stack()

更新 2:

截至目前，我们添加了所有元素将具有不同形状的约束，因此无法使用堆栈.如果它们都具有相同数量的维度和 dtype，您可以使用 typed_list，否则您需要自己修改 Theano 或展平输出列表.

As of now, we have the added constraint that all the elements will have different shapes, so stack can not be used. If they all have the same number of dimensions and dtype, you can use typed_list, otherwise you will need to modify Theano yourself or flatten the output's lists.

这篇关于Theano 多张量作为输出的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！