实现二进制交叉熵损失给出了与 Tensorflow 不同的答案 [英] Implementing Binary Cross Entropy loss gives different answer than Tensorflow's

问题描述

我正在使用 Raw python 实现二元交叉熵损失函数，但它给了我一个与 Tensorflow 截然不同的答案.这是我从 Tensorflow 得到的答案:-

将 numpy 导入为 np从 tensorflow.keras.losses 导入 BinaryCrossentropyy_true = np.array([1., 1., 1.])y_pred = np.array([1., 1., 0.])bce = BinaryCrossentropy()损失 = bce(y_true, y_pred)打印(损失.numpy())

输出:

<预><代码>>>>5.1416497230529785

据我所知，二元交叉熵的公式是这样的:

我用原始 python 实现了相同的如下:

def BinaryCrossEntropy(y_true, y_pred):m = y_true.shape[1]y_pred = np.clip(y_pred, 1e-7, 1 - 1e-7)# 计算损失损失 = -1/m * (np.dot(y_true.T, np.log(y_pred)) + np.dot((1 - y_true).T, np.log(1 - y_pred)))回波损耗打印(BinaryCrossEntropy(np.array([1, 1, 1]).reshape(-1, 1), np.array([1, 1, 0]).reshape(-1, 1)))

但是从这个函数我得到的损失值为:

<预><代码>>>>[[16.11809585]]

我怎样才能得到正确的答案?

解决方案

您的实施存在一些问题.这是正确的 numpy.

def BinaryCrossEntropy(y_true, y_pred):y_pred = np.clip(y_pred, 1e-7, 1 - 1e-7)term_0 = (1-y_true) * np.log(1-y_pred + 1e-7)term_1 = y_true * np.log(y_pred + 1e-7)返回-np.mean(term_0+term_1,axis=0)打印(BinaryCrossEntropy(np.array([1, 1, 1]).reshape(-1, 1),np.array([1, 1, 0]).reshape(-1, 1)))[5.14164949]

注意，在 tf.keras 模型训练，最好使用 keras 后端功能.您可以使用 keras 后端实用程序以同样的方式实现它.

def BinaryCrossEntropy(y_true, y_pred):y_pred = K.clip(y_pred, K.epsilon(), 1 - K.epsilon())term_0 = (1 - y_true) * K.log(1 - y_pred + K.epsilon())term_1 = y_true * K.log(y_pred + K.epsilon())返回-K.mean(term_0 + term_1，轴= 0)打印(二进制交叉熵(np.array([1., 1., 1.]).reshape(-1, 1),np.array([1., 1., 0.]).reshape(-1, 1)).numpy())[5.14164949]

I am implementing the Binary Cross-Entropy loss function with Raw python but it gives me a very different answer than Tensorflow. This is the answer I got from Tensorflow:-

import numpy as np
from tensorflow.keras.losses import BinaryCrossentropy

y_true = np.array([1., 1., 1.])
y_pred = np.array([1., 1., 0.])
bce = BinaryCrossentropy()
loss = bce(y_true, y_pred)
print(loss.numpy())

Output:

>>> 5.1416497230529785

From my Knowledge, the formula of Binary Cross entropy is this:

I implemented the same with raw python as follows:

def BinaryCrossEntropy(y_true, y_pred):
    m = y_true.shape[1]
    y_pred = np.clip(y_pred, 1e-7, 1 - 1e-7)
    # Calculating loss
    loss = -1/m * (np.dot(y_true.T, np.log(y_pred)) + np.dot((1 - y_true).T, np.log(1 - y_pred)))

    return loss

print(BinaryCrossEntropy(np.array([1, 1, 1]).reshape(-1, 1), np.array([1, 1, 0]).reshape(-1, 1)))

But from this function I get loss value to be:

>>> [[16.11809585]]

How can I get the right answer?

解决方案

There's some issue with your implementation. Here is the correct one with numpy.

def BinaryCrossEntropy(y_true, y_pred):
    y_pred = np.clip(y_pred, 1e-7, 1 - 1e-7)
    term_0 = (1-y_true) * np.log(1-y_pred + 1e-7)
    term_1 = y_true * np.log(y_pred + 1e-7)
    return -np.mean(term_0+term_1, axis=0)

print(BinaryCrossEntropy(np.array([1, 1, 1]).reshape(-1, 1), 
                         np.array([1, 1, 0]).reshape(-1, 1)))
[5.14164949]

Note, during the tf. keras model training, it's better to use keras backend functionality. You can implement it, in the same way, using the keras backend utilities.

def BinaryCrossEntropy(y_true, y_pred): 
    y_pred = K.clip(y_pred, K.epsilon(), 1 - K.epsilon())
    term_0 = (1 - y_true) * K.log(1 - y_pred + K.epsilon())  
    term_1 = y_true * K.log(y_pred + K.epsilon())
    return -K.mean(term_0 + term_1, axis=0)

print(BinaryCrossEntropy(
    np.array([1., 1., 1.]).reshape(-1, 1), 
    np.array([1., 1., 0.]).reshape(-1, 1)
    ).numpy())
[5.14164949]

这篇关于实现二进制交叉熵损失给出了与 Tensorflow 不同的答案的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！