在 tf.Variable 张量切片上分配 [英] assign on a tf.Variable tensor slice

问题描述

我正在尝试执行以下操作

I am trying to do the following

state[0,:] = state[0,:].assign( 0.9*prev_state + 0.1*( tf.matmul(inputs, weights) + biases ) )
for i in xrange(1,BATCH_SIZE):
    state[i,:] = state[i,:].assign( 0.9*state[i-1,:] + 0.1*( tf.matmul(inputs, weights) + biases ) )
prev_state = prev_state.assign( state[BATCH_SIZE-1,:] )

与

state = tf.Variable(tf.zeros([BATCH_SIZE, HIDDEN_1]), name='inner_state')
prev_state = tf.Variable(tf.zeros([HIDDEN_1]), name='previous_inner_state')

作为这个问题的后续.我收到一个错误，指出 Tensor 没有 assign 方法.

As a follow-up for this question. I get an error that Tensor does not have an assign method.

在 Variable 张量的切片上调用 assign 方法的正确方法是什么?

What is the correct way to call the assign method on a slice of a Variable tensor?

完整的当前代码:

import tensorflow as tf
import math
import numpy as np

INPUTS = 10
HIDDEN_1 = 20
BATCH_SIZE = 3


def create_graph(inputs, state, prev_state):
    with tf.name_scope('h1'):
        weights = tf.Variable(
        tf.truncated_normal([INPUTS, HIDDEN_1],
                            stddev=1.0 / math.sqrt(float(INPUTS))),
        name='weights')
        biases = tf.Variable(tf.zeros([HIDDEN_1]), name='biases')

        updated_state = tf.scatter_update(state, [0], 0.9 * prev_state + 0.1 * (tf.matmul(inputs[0,:], weights) + biases))
        for i in xrange(1, BATCH_SIZE):
          updated_state = tf.scatter_update(
              updated_state, [i], 0.9 * updated_state[i-1, :] + 0.1 * (tf.matmul(inputs[i,:], weights) + biases))

        prev_state = prev_state.assign(updated_state[BATCH_SIZE-1, :])
        output = tf.nn.relu(updated_state)
    return output

def data_iter():
    while True:
        idxs = np.random.rand(BATCH_SIZE, INPUTS)
        yield idxs

with tf.Graph().as_default():
    inputs = tf.placeholder(tf.float32, shape=(BATCH_SIZE, INPUTS))
    state = tf.Variable(tf.zeros([BATCH_SIZE, HIDDEN_1]), name='inner_state')
    prev_state = tf.Variable(tf.zeros([HIDDEN_1]), name='previous_inner_state')

    output = create_graph(inputs, state, prev_state)

    sess = tf.Session()
    # Run the Op to initialize the variables.
    init = tf.initialize_all_variables()
    sess.run(init)
    iter_ = data_iter()
    for i in xrange(0, 2):
        print ("iteration: ",i)
        input_data = iter_.next()
        out = sess.run(output, feed_dict={ inputs: input_data})

推荐答案

Tensorflow Variable 对象对更新切片的支持有限，使用 tf.scatter_update(), tf.scatter_add() 和 tf.scatter_sub() 操作.这些操作中的每一个都允许您指定一个变量、一个切片索引向量(表示变量第 0 维中的索引，表示要变异的连续切片)和一个值张量(表示要应用于的新值)变量，在相应的切片索引处).

Tensorflow Variable objects have limited support for updating slices, using the tf.scatter_update(), tf.scatter_add(), and tf.scatter_sub() ops. Each of these ops allows you to specify a variable, a vector of slice indices (representing indices in the 0th dimension of the variable, which indicate the contiguous slices to be mutated) and a tensor of values (representing the new values to be applied to the variable, at the corresponding slice indices).

要更新变量的单行，您可以使用 tf.scatter_update().例如，要更新 state 的第 0 行，您可以这样做:

To update a single row of the variable, you can use tf.scatter_update(). For example, to update the 0th row of state, you would do:

updated_state = tf.scatter_update(
    state, [0], 0.9 * prev_state + 0.1 * (tf.matmul(inputs, weights) + biases))

要链接多个更新，您可以使用从 tf.scatter_update() 返回的可变 updated_state 张量:

To chain multiple updates, you can use the mutable updated_state tensor that is returned from tf.scatter_update():

for i in xrange(1, BATCH_SIZE):
  updated_state = tf.scatter_update(
      updated_state, [i], 0.9 * updated_state[i-1, :] + ...)

prev_state = prev_state.assign(updated_state[BATCH_SIZE-1, :])

最后，您可以评估生成的 updated_state.op 以将所有更新应用到 state:

Finally, you can evaluate the resulting updated_state.op to apply all of the updates to state:

sess.run(updated_state.op)  # or `sess.run(updated_state)` to fetch the result

附注.您可能会发现使用 tf.scan() 计算中间状态，并将 prev_state 物化在一个变量中.

PS. You might find it more efficient to use tf.scan() to compute the intermediate states, and just materialize prev_state in a variable.

这篇关于在 tf.Variable 张量切片上分配的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！