Keras:加权二元交叉熵 [英] Keras: weighted binary crossentropy

问题描述

我试图用Keras实现加权二进制交叉熵，但是我不确定代码是否正确.培训输出似乎有些混乱.经过几个时期后，我得到的准确度约为0.15.我认为那太少了(即使是随机猜测).

I tried to implement a weighted binary crossentropy with Keras, but I am not sure if the code is correct. The training output seems to be a bit confusing. After a few epochs I just get an accuracy of ~0.15. I think thats much too less (even for a random guess).

输出中通常有大约11％的1和89％的零，因此权重分别为w_zero = 0.89和w_one = 0.11.

There are in general about 11% ones in the output and 89% zeros, therefore the weights are w_zero=0.89 and w_one=0.11.

我的代码:

def create_weighted_binary_crossentropy(zero_weight, one_weight):

    def weighted_binary_crossentropy(y_true, y_pred):

        # Original binary crossentropy (see losses.py):
        # K.mean(K.binary_crossentropy(y_true, y_pred), axis=-1)

        # Calculate the binary crossentropy
        b_ce = K.binary_crossentropy(y_true, y_pred)

        # Apply the weights
        weight_vector = y_true * one_weight + (1. - y_true) * zero_weight
        weighted_b_ce = weight_vector * b_ce

        # Return the mean error
        return K.mean(weighted_b_ce)

    return weighted_binary_crossentropy

也许有人看到错了吗?

谢谢

推荐答案

通常，少数群体的权重更高.最好使用one_weight=0.89, zero_weight=0.11(顺便说一句，您可以使用class_weight={0: 0.11, 1: 0.89}，如注释中所建议).

Normally, the minority class will have a higher class weight. It'll be better to use one_weight=0.89, zero_weight=0.11 (btw, you can use class_weight={0: 0.11, 1: 0.89}, as suggested in the comment).

在类不平衡下，您的模型看到的零比零多得多.它还将学会预测比零更多的零，因为这样做可以最大程度地减少训练损失.这就是为什么您看到的精度接近比例0.11的原因.如果对模型预测取平均值，则该平均值应该非常接近零.

Under class imbalance, your model is seeing much more zeros than ones. It will also learn to predict more zeros than ones because the training loss can be minimized by doing so. That's also why you're seeing an accuracy close to the proportion 0.11. If you take an average over model predictions, it should be very close to zero.

使用类权重的目的是更改损失函数，以使简单的解决方案"(即预测零)无法将训练损失最小化，这就是为什么最好使用更大的权重来降低训练损失的原因.一个.

The purpose of using class weights is to change the loss function so that the training loss cannot be minimized by the "easy solution" (i.e., predicting zeros), and that's why it'll be better to use a higher weight for ones.

请注意，最佳权重不一定是0.89和0.11.有时，您可能需要尝试采用对数或平方根(或满足one_weight > zero_weight的任何权重)进行计算.

Note that the best weights are not necessarily 0.89 and 0.11. Sometimes you might have to try something like taking logarithms or square roots (or any weights satisfying one_weight > zero_weight) to make it work.

这篇关于Keras:加权二元交叉熵的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！