使用 Scikit-learn 计算信息增益 [英] Information Gain calculation with Scikit-learn
问题描述
我正在使用 Scikit-learn 进行文本分类.我想针对(稀疏)文档项矩阵中的类计算每个属性的信息增益.
I am using Scikit-learn for text classification. I want to calculate the Information Gain for each attribute with respect to a class in a (sparse) document-term matrix.
信息增益定义为H(Class) - H(Class | Attribute)
,其中H
是熵.
The Information Gain is defined as H(Class) - H(Class | Attribute)
, where H
is the entropy.
使用weka,这可以通过信息增益属性.但是我在scikit-learn中没有找到这个度量.
Using weka, this can be accomplished with the InfoGainAttribute. But I haven't found this measure in scikit-learn.
然而,建议公式上述信息增益与互信息的度量相同.这也符合维基百科中的定义.
However, it has been suggested that the formula above for Information Gain is the same measure as mutual information. This matches also the definition in wikipedia.
是否可以在 scikit-learn 中使用特定的互信息设置来完成此任务?
Is it possible to use a specific setting for mutual information in scikit-learn to accomplish this task?
推荐答案
您可以使用 scikit-learn 的 mutual_info_classif
这是一个例子
You can use scikit-learn's mutual_info_classif
here is an example
from sklearn.datasets import fetch_20newsgroups
from sklearn.feature_selection import mutual_info_classif
from sklearn.feature_extraction.text import CountVectorizer
categories = ['talk.religion.misc',
'comp.graphics', 'sci.space']
newsgroups_train = fetch_20newsgroups(subset='train',
categories=categories)
X, Y = newsgroups_train.data, newsgroups_train.target
cv = CountVectorizer(max_df=0.95, min_df=2,
max_features=10000,
stop_words='english')
X_vec = cv.fit_transform(X)
res = dict(zip(cv.get_feature_names(),
mutual_info_classif(X_vec, Y, discrete_features=True)
))
print(res)
这将输出每个属性的字典,即词汇表中的项目作为键,它们的信息增益作为值
this will output a dictionary of each attribute, i.e. item in the vocabulary as keys and their information gain as values
这里是输出示例
{'bible': 0.072327479595571439,
'christ': 0.057293733680219089,
'christian': 0.12862867565281702,
'christians': 0.068511328611810071,
'file': 0.048056478042481157,
'god': 0.12252523919766867,
'gov': 0.053547274485785577,
'graphics': 0.13044709565039875,
'jesus': 0.09245436105573257,
'launch': 0.059882179387444862,
'moon': 0.064977781072557236,
'morality': 0.050235104394123153,
'nasa': 0.11146392824624819,
'orbit': 0.087254803670582998,
'people': 0.068118370234354936,
'prb': 0.049176995204404481,
'religion': 0.067695617096125316,
'shuttle': 0.053440976618359261,
'space': 0.20115901737978983,
'thanks': 0.060202010019767334}
这篇关于使用 Scikit-learn 计算信息增益的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!