比较 R、statmodels、sklearn 的分类任务与逻辑回归 [英] Comparison of R, statmodels, sklearn for a classification task with logistic regression

查看:26
本文介绍了比较 R、statmodels、sklearn 的分类任务与逻辑回归的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

我在 R、python statmodels 和 sklearn 中做了一些逻辑回归实验.虽然 R 和 statmodels 给出的结果一致,但与 sklearn 返回的结果存在一些差异.我想了解为什么这些结果不同.据我所知,这可能与木头下使用的优化算法不同.

I have made some experiments with logistic regression in R, python statmodels and sklearn. While the results given by R and statmodels agree, there is some discrepency with what is returned by sklearn. I would like to understand why these results are different. I understand that it is probably not the same optimization algorithms that are used under the wood.

具体来说,我使用标准的Default 数据集(在ISL 书籍中使用一>).以下 Python 代码将数据读入数据帧 Default.

Specifically, I use the standard Default dataset (used in the ISL book). The following Python code reads the data into a dataframe Default.

import pandas as pd
 # data is available here
Default = pd.read_csv('https://d1pqsl2386xqi9.cloudfront.net/notebooks/Default.csv', index_col=0)
 #
Default['default']=Default['default'].map({'No':0, 'Yes':1})
Default['student']=Default['student'].map({'No':0, 'Yes':1})
 #
I=Default['default']==0
print("Number of 'default' values :", Default[~I]['balance'].count())

默认"值的数量:333.

Number of 'default' values : 333.

总共有10000个例子,只有333个正例

There is a total of 10000 examples, with only 333 positives

我使用以下内容

library("ISLR")
data(Default,package='ISLR')
 #write.csv(Default,"default.csv")
glm.out=glm('default~balance+income+student', family=binomial, data=Default)
s=summary(glm.out)
print(s)
#
glm.probs=predict(glm.out,type="response") 
glm.probs[1:5]
glm.pred=ifelse(glm.probs>0.5,"Yes","No")
 #attach(Default)
t=table(glm.pred,Default$default)
print(t)
score=mean(glm.pred==Default$default)
print(paste("score",score))

结果如下

调用:glm(formula = "default~balance+income+student", family =二项式,数据 = 默认)

Call: glm(formula = "default~balance+income+student", family = binomial, data = Default)

偏差残差:最小 1Q 中值 3Q 最大值
-2.4691 -0.1418 -0.0557 -0.0203 3.7383

Deviance Residuals: Min 1Q Median 3Q Max
-2.4691 -0.1418 -0.0557 -0.0203 3.7383

系数:

Estimate Std. Error z value Pr(>|z|)        
(Intercept) -1.087e+01  4.923e-01 -22.080  < 2e-16 
balance      5.737e-03    2.319e-04  24.738  < 2e-16  
income       3.033e-06  8.203e-06   0.370   0.71152     
studentYes  -6.468e-01  2.363e-01  -2.738  0.00619  

(二项式族的色散参数取为 1)

(Dispersion parameter for binomial family taken to be 1)

Null deviance: 2920.6  on 9999  degrees of freedom Residual 

偏差:1571.5 on 9996 自由度 AIC:1579.5

deviance: 1571.5 on 9996 degrees of freedom AIC: 1579.5

Fisher Scoring 迭代次数:8

Number of Fisher Scoring iterations: 8

     glm.pred   No  Yes
 No  9627  228
 Yes   40  105 

1得分 0.9732"

我懒得剪切和粘贴用 statmodels 获得的结果.可以说它们与 R 给出的非常相似.

I am too lazy to cut and paste the results obtained with statmodels. It suffice to say that they are extremely similar to those given by R.

对于 sklearn,我运行了以下代码.

For sklearn, I ran the following code.

  • 有一个参数 class_weight 用于考虑不平衡的类.我测试了 class_weight=None(没有加权——我认为这是 R 中的默认值)和 class_weight='auto'(使用数据中发现的逆频率加权)
  • 我还把C=10000,正则化参数的倒数,以尽量减少正则化的影响.

~~

import sklearn
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import confusion_matrix

features = Default[[ 'balance', 'income' ]]
target = Default['default']
# 
for weight in (None,  'auto'):
    print("*"*40+"
weight:",weight)

    classifier = LogisticRegression(C=10000, class_weight=weight, random_state=42) 
                #C=10000 ~ no regularization

    classifier.fit(features, target,)  #fit classifier on whole base
    print("Intercept", classifier.intercept_)
    print("Coefficients", classifier.coef_)

    y_true=target
    y_pred_cls=classifier.predict_proba(features)[:,1]>0.5
    C=confusion_matrix(y_true,y_pred_cls)

    score=(C[0,0]+C[1,1])/(C[0,0]+C[1,1]+C[0,1]+C[1,0])
    precision=(C[1,1])/(C[1,1]+C[0 ,1])
    recall=(C[1,1])/(C[1,1]+C[1,0])
    print("
 Confusion matrix")
    print(C)
    print()
    print('{s:{c}<{n}}{num:2.4}'.format(s='Score',n=15,c='', num=score))
    print('{s:{c}<{n}}{num:2.4}'.format(s='Precision',n=15,c='', num=precision))
    print('{s:{c}<{n}}{num:2.4}'.format(s='Recall',n=15,c='', num=recall))

结果如下.

> **************************************** 
>weight: None 
>
>Intercept [ -1.94164126e-06] 
>
>Coefficients [[ 0.00040756 -0.00012588]]
> 
>  Confusion matrix 
>
>     [[9664    3]  
>     [ 333    0]]
> 
>     Score          0.9664 
>     Precision      0.0 
>     Recall         0.0
>
> **************************************** 
>weight: auto 
>
>Intercept [-8.15376429] 
>
>Coefficients 
>[[  5.67564834e-03   1.95253338e-05]]
> 
>  Confusion matrix 
>
>     [[8356 1311]  
>     [  34  299]]
> 
>     Score          0.8655 
>     Precision      0.1857 
>     Recall         0.8979

我观察到的是,对于 class_weight=None,Score 非常好,但 没有 正例被认可.准确率和召回率为零.发现的系数非常,尤其是截距.修改 C 不会改变事情.对于 class_weight='auto' 事情似乎更好,但我仍然有一个非常低的精度(太多正分类).同样,更改 C 无济于事.如果我手动修改截距,我可以恢复R给出的结果.所以我怀疑这两种情况下截距的估计之间存在差异.由于这对三态的规范有影响(类似于对脉冲的重新采样),这可以解释性能的差异.

What I observe is that for class_weight=None, the Score is excellent but no positive example is recognized. Precision and recall are at zero. The coefficients found are very small, particularly the intercept. Modifying C does not change things. For class_weight='auto' things seems better but I still have a precision which is very low (too much positive classified). Again, changing C does not help. If I modify the intercept by hand, I can recover the results given by R. So I suspect that here is a discrepency between the estimation of the intecepts in the two cases. As this has a consequence in the specification of the threeshold (analog to a resampling of pulations), this can explain the differences in performances.

但是,我欢迎任何有关在两种解决方案之间进行选择的建议,并有助于理解这些差异的根源.谢谢.

However, I would welcome any advice for the choice between the two solutions and help to understand the origin of these differences. Thanks.

推荐答案

我遇到了类似的问题,结果是 在/r/MachineLearning 上发布有关它的信息.事实证明,差异可以归因于数据标准化.如果数据标准化,无论 scikit-learn 使用何种方法来查找模型参数,都会产生更好的结果.scikit-learn 有一些讨论预处理数据(包括标准化)的文档,可以在这里找到.

I ran into a similar issue and ended up posting about it on /r/MachineLearning. It turns out the difference can be attributed to data standardization. Whatever approach scikit-learn is using to find the parameters of the model will yield better results if the data is standardized. scikit-learn has some documentation discussing preprocessing data (including standardization), which can be found here.

Number of 'default' values : 333
Intercept: [-6.12556565]
Coefficients: [[ 2.73145133  0.27750788]]

Confusion matrix
[[9629   38]
 [ 225  108]]

Score          0.9737
Precision      0.7397
Recall         0.3243

代码

# scikit-learn vs. R
# http://stackoverflow.com/questions/28747019/comparison-of-r-statmodels-sklearn-for-a-classification-task-with-logistic-reg

import pandas as pd
import sklearn

from sklearn.linear_model import LogisticRegression
from sklearn.metrics import confusion_matrix
from sklearn import preprocessing

# Data is available here.
Default = pd.read_csv('https://d1pqsl2386xqi9.cloudfront.net/notebooks/Default.csv', index_col = 0)

Default['default'] = Default['default'].map({'No':0, 'Yes':1})
Default['student'] = Default['student'].map({'No':0, 'Yes':1})

I = Default['default'] == 0
print("Number of 'default' values : {0}".format(Default[~I]['balance'].count()))

feats = ['balance', 'income']

Default[feats] = preprocessing.scale(Default[feats])

# C = 1e6 ~ no regularization.
classifier = LogisticRegression(C = 1e6, random_state = 42) 

classifier.fit(Default[feats], Default['default'])  #fit classifier on whole base
print("Intercept: {0}".format(classifier.intercept_))
print("Coefficients: {0}".format(classifier.coef_))

y_true = Default['default']
y_pred_cls = classifier.predict_proba(Default[feats])[:,1] > 0.5

confusion = confusion_matrix(y_true, y_pred_cls)
score = float((confusion[0, 0] + confusion[1, 1])) / float((confusion[0, 0] + confusion[1, 1] + confusion[0, 1] + confusion[1, 0]))
precision = float((confusion[1, 1])) / float((confusion[1, 1] + confusion[0, 1]))
recall = float((confusion[1, 1])) / float((confusion[1, 1] + confusion[1, 0]))
print("
Confusion matrix")
print(confusion)
print('
{s:{c}<{n}}{num:2.4}'.format(s = 'Score', n = 15, c = '', num = score))
print('{s:{c}<{n}}{num:2.4}'.format(s = 'Precision', n = 15, c = '', num = precision))
print('{s:{c}<{n}}{num:2.4}'.format(s = 'Recall', n = 15, c = '', num = recall))

这篇关于比较 R、statmodels、sklearn 的分类任务与逻辑回归的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!

查看全文
登录 关闭
扫码关注1秒登录
发送“验证码”获取 | 15天全站免登陆