为什么 MNLogit 返回 `classes_num - 1` 参数以及如何获取它们? [英] Why MNLogit returns `classes_num - 1` params and how get them all?

问题描述

如果我的类很少，即 3.我希望得到 3 个广义线性回归系数数组，如 sklearn.linear_model.LogisticRegression 但 statsmodels.discrete.discrete_model.MNLogit 提供 classes_num - 1 个系数(在本例中为 -- 2).

If I have few classes, i.e. 3. I am expecting to get 3 Generalized Linear Regression coefficients arrays, as in sklearn.linear_model.LogisticRegression but statsmodels.discrete.discrete_model.MNLogit provides classes_num - 1 coefficients (in this case -- 2).

示例:

import statsmodels.api as st
from sklearn.linear_model import LogisticRegression


iris = st.datasets.get_rdataset('iris','datasets')

y = iris.data.Species
x = iris.data.iloc[:, :-1]
mdl = st.MNLogit(y, x)
# mdl_fit = mdl.fit()
mdl_fit = mdl.fit(method='bfgs' , maxiter=1000)
print(mdl_fit.params.shape)  # (4, 2)

model = LogisticRegression(fit_intercept = False, C = 1e9)
mdl = model.fit(x, y)
print(model.coef_.shape)  # (3, 4)

我应该如何使用 MNLogit 获得所有 3 个类的回归系数?

How I am supposed to get regression coefficients for all 3 classes using MNLogit?

推荐答案

这些系数不会被计算来强制模型的可识别性.换句话说，不计算它们可确保其他类的系数唯一.如果您有三组系数，则有无数模型给出相同的预测，但系数值不同.如果您想知道标准误差、p 值等，这很糟糕.

Those coefficients are not computed to force identifiability of the model. In other words, not computing them ensures that the coefficients for the other classes are unique. If you had three sets of coefficients, there are an infinite number of models that give the same predictions but have different values for the coefficients. And this is bad if you want to know standard errors, p-values and so on.

假设缺失类的 logit 为零.演示:

The logit of the missing class is assumed to be zero. Demo:

mm = st.MNLogit(
    np.random.randint(1, 5, size=(100,)),
    np.random.normal(size=(100, 3))
)

res = mm.fit()
xt = np.random.normal(size=(2, 3))
res.predict(xt)

结果:

array([[0.19918096, 0.34265719, 0.21307297, 0.24508888],
       [0.33974178, 0.21649687, 0.20971884, 0.23404251]])

现在这些是对数，加上第一类的零

Now these are the logits, plus the zeros for the first class

logits = np.hstack([np.zeros((xt.shape[0], 1)), xt.dot(res.params)])

array([[ 0.        ,  0.54251673,  0.06742093,  0.20740715],
       [ 0.        , -0.45060978, -0.4824181 , -0.37268309]])

以及通过 softmax 的预测:

And the predictions via a softmax:

np.exp(logits) / (np.sum(np.exp(logits), axis=1, keepdims=1))

array([[0.19918096, 0.34265719, 0.21307297, 0.24508888],
       [0.33974178, 0.21649687, 0.20971884, 0.23404251]])

与模型的预测相匹配.

重申:你找不到那些系数.对第一类使用常数 logit 为零.而且您无法找到对第一类有多大影响的特征.这实际上是一个不适定的问题:特征不能对参考类产生影响，因为参考类从来没有被直接预测过.系数告诉您的是，与参考类相比，给定类的对数优势因特定特征的单位增加而变化多少.

To reiterate: you cannot find those coefficients. Use a constant logit of zero for the first class. And you cannot find how much features are influential for the first class. That is actually an ill-posed question: the features cannot be influential for the reference class, because the reference class is never predicted directly. What the coefficients tell you is how much the log-odds for a given class, compared the reference class, change as a result of an unit increase for a particular feature.

这篇关于为什么 MNLogit 返回 `classes_num - 1` 参数以及如何获取它们?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！