为什么我获得负面信息收益？ [英] Why am I getting a negative information gain?

问题描述

[已解决]

我的错误是我没有意识到如果所有熵都是一种类型，那么熵就是0。因此，如果全部为正，则熵为0，如果全部为负，则熵也为零。如果等量的正负相等，则熵将为1。

My mistake was that I did not realise that entropy is 0 if all are of one type. Thus if all are positive, entropy is 0 and if all are negative it is zero as well. Entropy will be 1 if equal amount are positive and negative.

一个人获得负信息增益没有意义。

It does not make sense that one would get negative information gain.

但是根据此示例，我获得了负信息增益。

However based on this example I am getting a negative information gain.

以下是数据：

here is the data:

如果我计算了Humidity属性的信息增益，我会得到

And if I calculate the information gain on the Humidity attribute I get this:

很显然我在这里错过了一些东西。

Obviously I am missing something here.

编辑：
阐明我的理解。

To clarify how I understand it.

整个系统的熵定义为：

在这种情况下为：

每个属性的信息增益定义为：

And the information gain per atribute is defined as:

对于湿度，我计算得出：

Which for humidity I calculate to:

系统的熵-（1 / 4）湿度熵正常-（3/4）湿度熵高

Entropy of system - (1/4)Entropy of Humidity Normal - (3/4)Entropy of Humidity High

根据此Libre Office Calc：

As per this Libre Office Calc:

或者是我对属性信息获取公式的理解

Or is my understanding of the formula for information gain for an attribute incorrect?

推荐答案

首先，我假设您的 S 变量为 EnjoySport 。 （我想您可以更清楚地表达文字，顺便说一句。）

To begin with, I'm assuming your S variable is EnjoySport. (I think you could phrase the text more clearly, BTW.)

因此， S 的熵为0.8113，但这是最后一部分我同意。

So the entropy of S is 0.8113, but that's the last part with which I agree.

给定 Normal 的 S 的熵为0，因为它是确定性的。

The entropy of S given Normal is 0, as it is deterministic.

给定 High 的 S 的熵为0.91829583405448945，但是您需要将其乘以0.75，因为这是 Normal的概率。这样就可以得到0.68872187554086706。

The entropy of S given High is 0.91829583405448945, but you need to multiply that by 0.75, because that is the probability of Normal. So that gives you 0.68872187554086706.

差异与预期的一样是非负的。

The difference is non-negative, as expected.

请注意，信息收益是熵的预期差异，并且期望需要考虑条件事件的概率。

Note that the Information gain is the expected difference in Entropy, and the expectation needs to take into account the probability of the conditioned event.

这篇关于为什么我获得负面信息收益？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！