TensorFlow的Print或K.print_tensor不在损失函数中打印中间张量 [英] TensorFlow's Print or K.print_tensor are not printing intermediate tensors in loss function

问题描述

我为Keras模型编写了一个相当复杂的损失函数，并且在训练过程中一直保持返回nan的作用.因此，我需要在训练时打印中间张量.我知道您无法在损失函数中执行K.eval，因为张量未初始化.但是，我已经尝试了K.print_tensor()和tf.Print()，但都没有用.

I have written a rather complex loss function for a Keras model and it keeps returning nan while training. Therefore, I need to print the intermediate tensors while training. I understand that you cannot do K.eval in your loss function because the tensors are not initialized. However, I have tried both K.print_tensor() and tf.Print() and neither work.

我几乎想做这样的事情:

Pretty much I want to do something like this:

def mean_squared_error(y_true, y_pred):
    print("mean_squared_error")
    loss = K.mean(K.square(y_pred - y_true), axis=-1)
    loss = tf.Print(loss, [loss])
    return loss
model.compile(optimizer=self.optimizer, loss=mean_squared_error)

在实践中，我将mean_squared_error替换为我的自定义损失. "mean_squared_error"将被打印，但不是我尝试使用TensorFlow打印(也不是Keras打印)打印的值.我还尝试了与在Keras培训期间如何在损失函数中打印完全相同的代码?在控制台上看不到任何内容.

In practice, I would replace mean_squared_error with my custom loss. "mean_squared_error" would get printed, but not the values I try to print using TensorFlow print (nor Keras print). I also tried the exact same code as in How do I print inside the loss function during training in Keras? and I still don't see anything getting printed in the console.

此外，我还编写了一个单独的文件来测试某些内容.

In addition, I have written a separate file to test something.

import tensorflow as tf
import keras.backend as K

input1 = K.constant(1)
input2 = K.constant(2)
input3 = K.constant(3)

node1 = tf.add(input1, input2)
print_output = K.print_tensor(node1)
output = tf.multiply(print_output, input3)

也没有打印任何内容.

Nothing gets printed either.

我错误地使用了TensorFlow的Print和Keras print_tensor吗?还是将结果打印在其他地方?我尝试使用print("test", file=sys.stderr)测试控制台的stderr，并获得正确的输出test.

Am I using TensorFlow's Print and Keras print_tensor wrongly? Or are the results printed elsewhere? I have tried to test for my console's stderr using print("test", file=sys.stderr) and got the correct output test.

为清楚起见，我知道您可以使用K.eval来使测试代码打印出张量的值，但是由于我无法在损失函数中使用K.eval，因此我需要将tf.Print或工作.

For clarification, I know that you can use K.eval to make the test code print out values of the tensor, but since I cannot use K.eval in my loss function, I need to make tf.Print or K.print_tensor work.

推荐答案

这里的问题是训练代码通常实际上并不依赖于损失张量的值！通常，您无需计算损失的实际值就可以计算损失的梯度，这意味着tensorflow的运行时可以自由地从图中修剪损失的实际执行情况.

The issue here is that the training code often does not actually depend on the value of the loss tensor! Usually you can compute the gradient of a loss without ever computing the actual value of the loss, and this means tensorflow's runtime is free to prune the actual execution of the loss from the graph.

您可以将损失函数包装在 tf.contrib.eager中.defun 装饰器，它的副作用是即使您的向后传递不需要所有有状态的操作，它们也可以运行.

You can wrap your loss function in a tf.contrib.eager.defun decorator, which has the side effect of guaranteeing that all stateful ops in your function run even if they are not needed by the backward pass.

这篇关于TensorFlow的Print或K.print_tensor不在损失函数中打印中间张量的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！