tensorflow:在tf.map_fn的fn中创建变量返回值错误 [英] tensorflow: creating variables in fn of tf.map_fn returns value error
问题描述
我对map_fn中的变量初始化有疑问.
I have questions regarding variable initialization in map_fn.
我试图在张量中的每个单独元素上分别应用一些高速公路图层,因此我认为map_fn可能是实现此目标的最佳方法.
I was trying to apply some highway layers separately on each individual element in a tensor, so i figure map_fn might be the best way to do it.
segment_list = tf.reshape(raw_segment_embedding,[batch_size*seqlen,embed_dim])
segment_embedding = tf.map_fn(lambda x: stack_highways(x, hparams), segment_list)
现在的问题是我的fn,即stack_highways,创建变量,并且由于某种原因,tensorflow无法初始化这些变量并给出此错误.
Now the problem is my fn, i.e. stack_highways, create variables, and for some reason tensorflow fails to initialize those variables and give this error.
W = tf.Variable(tf.truncated_normal(W_shape, stddev=0.1), name='weight')
ValueError: Initializer for variable body/model/parallel_0/body/map/while/highway_layer0/weight/ is from inside a control-flow construct, such as a loop or conditional. When creating a variable inside a loop or conditional, use a lambda as the initializer.
基于错误,我现在很无能为力,我认为它与范围无关,但是我不知道如何使用lambda作为初始化程序(我什至不知道这到底是什么意思).下面是stack_highways的实现,任何建议将不胜感激.
I am pretty clueless now, based on the error I suppose it is not about scope but I have no idea how to use a lambda as the initializer (I dont even know what exactly does that mean). Below are the implementation of stack_highways, any advice would be much appreciated..
def weight_bias(W_shape, b_shape, bias_init=0.1):
"""Fully connected highway layer adopted from
https://github.com/fomorians/highway-fcn/blob/master/main.py
"""
W = tf.Variable(tf.truncated_normal(W_shape, stddev=0.1), name='weight')
b = tf.Variable(tf.constant(bias_init, shape=b_shape), name='bias')
return W, b
def highway_layer(x, size, activation, carry_bias=-1.0):
"""Fully connected highway layer adopted from
https://github.com/fomorians/highway-fcn/blob/master/main.py
"""
W, b = weight_bias([size, size], [size])
with tf.name_scope('transform_gate'):
W_T, b_T = weight_bias([size, size], bias_init=carry_bias)
H = activation(tf.matmul(x, W) + b, name='activation')
T = tf.sigmoid(tf.matmul(x, W_T) + b_T, name='transform_gate')
C = tf.sub(1.0, T, name="carry_gate")
y = tf.add(tf.mul(H, T), tf.mul(x, C), name='y') # y = (H * T) + (x * C)
return y
def stack_highways(x, hparams):
"""Create highway networks, this would not create
a padding layer in the bottom and the top, it would
just be layers of highways.
Args:
x: a raw_segment_embedding
hparams: run hyperparameters
Returns:
y: a segment_embedding
"""
highway_size = hparams.highway_size
activation = hparams.highway_activation #tf.nn.relu
carry_bias_init = hparams.highway_carry_bias
prev_y = None
y = None
for i in range(highway_size):
with tf.name_scope("highway_layer{}".format(i)) as scope:
if i == 0: # first, input layer
prev_y = highway_layer(x, highway_size, activation, carry_bias=carry_bias_init)
elif i == highways - 1: # last, output layer
y = highway_layer(prev_y, highway_size, activation, carry_bias=carry_bias_init)
else: # hidden layers
prev_y = highway_layer(prev_y, highway_size, activation, carry_bias=carry_bias_init)
return y
最热烈的问候,科尔曼
推荐答案
TensorFlow提供了两种初始化变量的主要方法:
TensorFlow provides two main ways of initializing variables:
- "lambda"初始值设定项:返回初始值的可调用项.TF提供了许多包装精美的一个.
- 通过张量值初始化:这是您当前正在使用的.
错误消息指出,在 while_loop
(内部由 map_fn
调用)中使用变量时,您需要使用第一种类型的初始化程序.(一般而言,lambda初始化程序对我来说似乎更健壮.)
The error message is stating that you need to use the first type of initializer when using variables from within a while_loop
(which map_fn
calls internally). (In general lambda initializers seem more robust to me.)
此外,在过去, tf.get_variable似乎比tf.Variable更可取.控制流.
因此,我怀疑您可以通过将 weight_bias
函数修复为以下内容来解决您的问题:
So, I suspect you can resolve your issue by fixing your weight_bias
function to something like this:
def weight_bias(W_shape, b_shape, bias_init=0.1):
"""Fully connected highway layer adopted from
https://github.com/fomorians/highway-fcn/blob/master/main.py
"""
W = tf.get_variable("weight", shape=W_shape,
initializer=tf.truncated_normal_initializer(stddev=0.1))
b = tf.get_variable("bias", shape=b_shape,
initializer=tf.constant_inititializer(bias_init))
return W, b
希望有帮助!
这篇关于tensorflow:在tf.map_fn的fn中创建变量返回值错误的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!