在每个训练步骤中能否获得目标函数值? [英] Is it possible to get the objective function value during each training step?
问题描述
在通常的TensorFlow训练循环中,例如
In the usual TensorFlow training loop, such as
train_op = tf.train.AdamOptimizer().minimize(cross_entropy)
with tf.Session() as sess:
for i in range(num_steps):
# ...
train_op.run(feed_dict = feed_dict)
train_op.run
返回None
.
但是,有时收集中间结果(例如物镜的值或准确性)很有用.
However, sometimes it's useful to collect intermediate results, such as the value of the objective or the accuracy.
添加额外的sess.run
调用将需要再次进行前向传播,从而增加了运行时间:
Adding extra sess.run
calls would require doing the forward propagation again, increasing the run time:
train_op = tf.train.AdamOptimizer().minimize(cross_entropy)
with tf.Session() as sess:
for i in range(num_steps):
# ...
o, a = sess.run([objective, accuracy], feed_dict = feed_dict)
train_op.run(feed_dict = feed_dict)
是否可以一次性在TensorFlow中做到这一点?
Is it possible to do this in TensorFlow in one go?
有人建议
sess.run([objective, accuracy, train_op], feed_dict = feed_dict)
但是结果取决于列表元素的执行顺序:
but the results depend on the execution order of the list elements:
[objective, accuracy, train_op]
似乎是不确定的-根据是否使用CUDA,您将获得不同的结果.
推荐答案
只需将train_op
添加到要评估的节点列表中即可.
Simply add you train_op
to the list of nodes to be evaluated.
o, a, _ = sess.run([objective, accuracy, train_op], feed_dict = feed_dict)
关于训练步骤及其在评估中的顺序,我做了以下小实验:
Regarding the training step and its order in the evaluation, I made the following small experiment:
import tensorflow as tf
x = tf.Variable(0, dtype=tf.float32)
loss = tf.nn.l2_loss(x-1)
train_opt = tf.train.GradientDescentOptimizer(1)
train_op = train_opt.minimize(loss)
init_op = tf.global_variables_initializer()
sess = tf.Session()
sess.run(init_op)
x_val, _, loss_val = sess.run([x, train_op, loss])
# returns x_val = 1.0, loss_val = 0.5
这种情况比我最初想象的要混乱.似乎可以肯定的是,访存的执行顺序并不取决于它们在列表中的位置:x_val
和loss_val
将是相同的,而与它们在列表中的位置无关.
The situation is more confused than I initially thought. What seems to be a given is that the order of execution of the fetches does not depend of their respective position in the list: x_val
and loss_val
will be the same independently of their position in the list.
但是,正如@MaxB注意到的那样,不能保证它们的执行顺序.在GPU上运行上述代码时,x_val
设置为0.0(初始值).但是,在CPU上运行时,x_val
为1.0,即从train_op
更新后的 值.
However, as @MaxB noticed, their order of execution is not guaranteed. When running the above code on GPU, x_val
is set to 0.0, the initial value. However, when running on CPU, x_val
is 1.0, that is, the value after the update from train_op
.
如上面的实验所示,这种依赖于配置的行为可能仅限于由训练操作更新的变量,但不能保证它们来自tf的文档.
This configuration-dependant behavior could be limited to variables updated by training operations, as the experiment above suggests, but their is no guarantee coming from tf's documentation.
这篇关于在每个训练步骤中能否获得目标函数值?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!