如何扩展 Tensorflow 变量 [英] How to expand a Tensorflow Variable

问题描述

有没有办法让 Tensorflow 变量变大?比如，假设我想在训练过程中向神经网络的一层添加一个神经元.我该怎么做?这个问题中的答案告诉我如何更改变量的形状，将其扩展以适应另一行权重，但我不知道如何初始化这些新权重.

Is there any way to make a Tensorflow Variable larger? Like, let's say I wanted to add a neuron to a layer of a neural network in the middle of training. How would I go about doing that? An answer in This question told me how to change the shape of the variable, to expand it to fit another row of weights, but I don't know how to initialize those new weights.

我想另一种方法可能涉及组合变量，例如首先在第二个变量中初始化权重，然后将其添加为第一个变量的新行或列，但我找不到任何让我也这样做.

I figure another way of going about this might involve combining variables, as in initializing the weights first in a second variable and then adding that in as a new row or column of the first variable, but I can't find anything that lets me do that either.

推荐答案

想通了.这是一个迂回的过程，但它是我能说的唯一一个真正起作用的过程.您需要先解包变量，然后将新变量附加到末尾，然后将它们重新打包在一起.

Figured it out. It's kind of a roundabout process, but it's the only one I can tell that actually functions. You need to first unpack the variables, then append the new variable to the end, then pack them back together.

如果您沿第一个维度展开，它会很短:只有 7 行实际代码.

If you're expanding along the first dimension, it's rather short: only 7 lines of actual code.

#the first variable is 5x3
v1 = tf.Variable(tf.zeros([5, 3], dtype=tf.float32), "1")

#the second variable is 1x3
v2 = tf.Variable(tf.zeros([1, 3], dtype=tf.float32), "2")

#unpack the first variable into a list of size 3 tensors
#there should be 5 tensors in the list
change_shape = tf.unpack(v1)

#unpack the second variable into a list of size 3 tensors
#there should be 1 tensor in this list
change_shape_2 = tf.unpack(v2)

#for each tensor in the second list, append it to the first list
for i in range(len(change_shape_2)):
  change_shape.append(change_shape_2[i])

#repack the list of tensors into a single tensor
#the shape of this resultant tensor should be [6, 3]
final = tf.pack(change_shape)

如果你想沿着第二个维度展开，它会变长一些.

If you want to expand along the second dimension, it gets somewhat longer.

#First variable, 5x3
v3 = tf.Variable(tf.zeros([5, 3], dtype=tf.float32))

#second variable, 5x1
v4 = tf.Variable(tf.zeros([5, 1], dtype=tf.float32))

#unpack tensors into lists of size 3 tensors and size 1 tensors, respectively
#both lists will hold 5 tensors
change = tf.unpack(v3)
change2 = tf.unpack(v4)

#for each tensor in the first list, unpack it into its own list
#this should make a 2d array of size 1 tensors, array will be 5x3
changestep2 = []
for i in range(len(change)):
  changestep2.append(tf.unpack(change[i]))

#do the same thing for the second tensor
#2d array of size 1 tensors, array will be 5x1
change2step2 = []
for i in range(len(change2)):
  change2step2.append(tf.unpack(change2[i]))

  #for each tensor in the array, append it onto the corresponding array in the first list
  for j in range(len(change2step2[i])):
    changestep2[i].append(change2step2[i][j])

  #pack the lists in the array back into tensors
  changestep2[i] = tf.pack(changestep2[i])

#pack the list of tensors into a single tensor
#the shape of this resultant tensor should be [5, 4]
final2 = tf.pack(changestep2)

我不知道是否有更有效的方法来做到这一点，但就目前而言，这是有效的.如有必要，更改更多维度将需要更多层列表.

I don't know if there's a more efficient way of doing this, but this works, as far as it goes. Changing further dimensions would require more layers of lists, as necessary.

这篇关于如何扩展 Tensorflow 变量的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！