使用Scikit-learn计算信息增益 [英] Information Gain calculation with Scikit-learn

问题描述

我使用Scikit学习的文本分类.我要计算的信息增益用于在(稀疏)文档术语矩阵相对于每个属性的一类. 信息增益被定义为H(类) - H(类|属性)，其中H是熵.

I am using Scikit-learn for text classification. I want to calculate the Information Gain for each attribute with respect to a class in a (sparse) document-term matrix. The Information Gain is defined as H(Class) - H(Class | Attribute), where H is the entropy.

使用weka可以通过 InfoGainAttribute实现.但是我还没有发现这一措施在scikit学习.

Using weka, this can be accomplished with the InfoGainAttribute. But I haven't found this measure in scikit-learn.

然而，它已经建议该公式对于以上信息增益是相同的措施，因为互信息.此比赛也维基中的定义.

However, it has been suggested that the formula above for Information Gain is the same measure as mutual information. This matches also the definition in wikipedia.

是否可以使用特定的设置相互信息scikit学习来完成这个任务?

Is it possible to use a specific setting for mutual information in scikit-learn to accomplish this task?

推荐答案

您可以使用scikit-learn的 这里是一个例子

You can use scikit-learn's mutual_info_classif here is an example

from sklearn.datasets import fetch_20newsgroups
from sklearn.feature_selection import mutual_info_classif
from sklearn.feature_extraction.text import CountVectorizer

categories = ['talk.religion.misc',
              'comp.graphics', 'sci.space']
newsgroups_train = fetch_20newsgroups(subset='train',
                                      categories=categories)

X, Y = newsgroups_train.data, newsgroups_train.target
cv = CountVectorizer(max_df=0.95, min_df=2,
                                     max_features=10000,
                                     stop_words='english')
X_vec = cv.fit_transform(X)

res = dict(zip(cv.get_feature_names(),
               mutual_info_classif(X_vec, Y, discrete_features=True)
               ))
print(res)

此将输出每个属性的词典，即在词汇作为键项目和它们的增益信息作为值

this will output a dictionary of each attribute, i.e. item in the vocabulary as keys and their information gain as values

这里是输出

{'bible': 0.072327479595571439,
 'christ': 0.057293733680219089,
 'christian': 0.12862867565281702,
 'christians': 0.068511328611810071,
 'file': 0.048056478042481157,
 'god': 0.12252523919766867,
 'gov': 0.053547274485785577,
 'graphics': 0.13044709565039875,
 'jesus': 0.09245436105573257,
 'launch': 0.059882179387444862,
 'moon': 0.064977781072557236,
 'morality': 0.050235104394123153,
 'nasa': 0.11146392824624819,
 'orbit': 0.087254803670582998,
 'people': 0.068118370234354936,
 'prb': 0.049176995204404481,
 'religion': 0.067695617096125316,
 'shuttle': 0.053440976618359261,
 'space': 0.20115901737978983,
 'thanks': 0.060202010019767334}

这篇关于使用Scikit-learn计算信息增益的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！