在Keras(Tensorflow后端)中使用binary_crossentropy损失 [英] Using binary_crossentropy loss in Keras (Tensorflow backend)
问题描述
在Keras文档中的培训示例中,
In the training example in Keras documentation,
https://keras.io/getting-started/sequential-model -guide/#training
binary_crossentropy ,并在网络的最后一层添加 Sigmoid 激活,但是是否有必要在最后一层添加 Sigmoid ?正如我在源代码中找到的:
binary_crossentropy is used and sigmoid activation is added in the network's last layer, but is it necessary that add sigmoid in the last layer? As I found in the source code:
def binary_crossentropy(output, target, from_logits=False):
"""Binary crossentropy between an output tensor and a target tensor.
Arguments:
output: A tensor.
target: A tensor with the same shape as `output`.
from_logits: Whether `output` is expected to be a logits tensor.
By default, we consider that `output`
encodes a probability distribution.
Returns:
A tensor.
"""
# Note: nn.softmax_cross_entropy_with_logits
# expects logits, Keras expects probabilities.
if not from_logits:
# transform back to logits
epsilon = _to_tensor(_EPSILON, output.dtype.base_dtype)
output = clip_ops.clip_by_value(output, epsilon, 1 - epsilon)
output = math_ops.log(output / (1 - output))
return nn.sigmoid_cross_entropy_with_logits(labels=target, logits=output)
Keras在Tensorflow中调用 sigmoid_cross_entropy_with_logits ,但在 sigmoid_cross_entropy_with_logits 函数中,再次计算
Keras invokes sigmoid_cross_entropy_with_logits in Tensorflow, but in sigmoid_cross_entropy_with_logits function, sigmoid(logits) is calculated again.
https://www.tensorflow.org/versions/master/api_docs/python/tf/nn/sigmoid_cross_entropy_with_logits
因此,我认为最后添加 Sigmoid 并不有意义,但是似乎我在网上发现的Keras中所有的二进制/多标签分类示例和教程都添加了 sigmoid .而且我不明白什么是
So I don't think it makes sense that add a sigmoid at last, but seemingly all the binary/multi-label classification examples and tutorials in Keras I found online added sigmoid at last. Besides I don't understand what is the meaning of
# Note: nn.softmax_cross_entropy_with_logits
# expects logits, Keras expects probabilities.
为什么Keras期望概率?它不使用 nn.softmax_cross_entropy_with_logits 函数吗?有道理吗?
Why Keras expects probabilities? Doesn't it use the nn.softmax_cross_entropy_with_logits function? Does it make sense?
谢谢.
推荐答案
您是对的,这就是正在发生的事情.我相信这是由于历史原因.
You're right, that's exactly what's happening. I believe this is due to historical reasons.
Keras是在张量流之前创建的,用作theano的包装.并且在theano中,必须手动计算Sigmax/softmax,然后应用交叉熵损失函数. Tensorflow可以在一个融合的操作中完成所有操作,但是社区已经采用了具有Sigmoid/softmax层的API.
Keras was created before tensorflow, as a wrapper around theano. And in theano, one has to compute sigmoid/softmax manually and then apply cross-entropy loss function. Tensorflow does everything in one fused op, but the API with sigmoid/softmax layer was already adopted by the community.
如果要避免不必要的logit<->概率转换,请用from_logits=True
调用binary_crossentropy
损失,并且不要添加S形层.
If you want to avoid unnecessary logit <-> probability conversions, call binary_crossentropy
loss withfrom_logits=True
and don't add the sigmoid layer.
这篇关于在Keras(Tensorflow后端)中使用binary_crossentropy损失的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!