Tensorflow，在 Tensorflow 的 sparse_categorical_crossentropy 中 from_logits = True 或 False 是什么意思? [英] Tensorflow, what does from_logits = True or False mean in sparse_categorical_crossentropy of Tensorflow?

问题描述

在Tensorflow 2.0中，有一个损失函数叫做

In Tensorflow 2.0, there is a loss function called

tf.keras.losses.sparse_categorical_crossentropy(labels, targets, from_logits = False)

我可以问一下设置from_logits = True或False有什么区别?我的猜测是，当传入值是logits时，您将from_logits设置为True，并且如果传入值是概率(由softmax等输出)，则只需将from_logits = False(这是默认设置)设置即可.

Can I ask you what are the differences between setting from_logits = True or False? My guess was that when incoming values are logits, you set from_logits = True, and if incoming values are probabilities(output by softmax etc.) then you just set from_logits = False (which is a default setting).

但是为什么呢?损失只是一些计算.为什么它需要根据其传入值而有所不同?我也在Google的Tensorflow教程中看到了 https://www.tensorflow.org/alpha/tutorials/sequences/text_generation 即使最后一层的传入值是logits，也不会设置from_logits = True.这是代码

But why? loss is just some calculation. Why does it need to differ by its incoming values? I also saw in google's tensorflow tutorial https://www.tensorflow.org/alpha/tutorials/sequences/text_generation that it doesnt set from_logits = True even if incoming values of the last layer are logits. Here is the code

@tf.function
def train_step(inp, target):
  with tf.GradientTape() as tape:
    predictions = model(inp)
    loss = tf.reduce_mean(
        tf.keras.losses.sparse_categorical_crossentropy(target, predictions))
  grads = tape.gradient(loss, model.trainable_variables)
  optimizer.apply_gradients(zip(grads, model.trainable_variables))

  return loss

模型所在的地方

 model = tf.keras.Sequential([
    tf.keras.layers.Embedding(vocab_size, embedding_dim, 
                              batch_input_shape=[batch_size, None]),
    tf.keras.layers.LSTM(rnn_units, 
                        return_sequences=True, 
                        stateful=True, 
                        recurrent_initializer='glorot_uniform'),
    tf.keras.layers.Dense(vocab_size)
  ])

其中没有softmax的最后一层.(此外，在本教程的另一部分中，它设置from_logits = True)

which does not have the last layer of softmax. (Also, in another part of the tutorial, it set from_logits = True)

那么，我是否将其设置为True都没关系吗?

So, doesn't it matter whether I set it True or not?

推荐答案

Deepak 提到的帖子有一些数学背景.

The post Deepak mentioned has some math background.

但是为了简单起见，from_logits = True表示crossEntropy层的输入是正常张量/logits，而如果from_logits = False，则表示输入是一个概率，通常您应该在最后一层中有一些softmax激活.

But for simplicity, from_logits=True means the input to crossEntropy layer is normal tensor/logits, while if from_logits=False, means the input is a probability and usually you should have some softmax activation in your last layer.

这篇关于Tensorflow，在 Tensorflow 的 sparse_categorical_crossentropy 中 from_logits = True 或 False 是什么意思?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！