如何获得具有预处理和分类步骤的决策树管道的特征重要性？ [英] How do I get feature importances for decision tree pipeline that has preprocessing and classification steps?

问题描述

我正在尝试将决策树模型适用于UCI成人数据集。为此，我构建了以下管道：

nominal_features = ['workclass', 'education', 'marital-status', 'occupation', 
                'relationship', 'race', 'sex', 'native-country']

nominal_transformer = Pipeline(steps=[
    ('imputer', SimpleImputer(strategy='most_frequent')),
    ('ohe', OneHotEncoder(handle_unknown='ignore'))
])

numeric_features = ['age', 'fnlwgt', 'capital-gain', 'capital-loss', 'hours-per-week']

numeric_transformer = Pipeline(steps=[
    ('scaler', StandardScaler())
])

preprocessor = ColumnTransformer(
    transformers=[
        ('numeric', numeric_transformer, numeric_features),
        ('nominal', nominal_transformer, nominal_features)
    ]) # remaining columns will be dropped by default

clf = Pipeline(steps=[
    ('preprocessor', preprocessor),
    ('classifier', DecisionTreeClassifier(criterion='entropy', random_state=0))
])

然后我通过调用

来适应我的模型

clf.fit(X_train, y_train)

然后，当我尝试获取功能重要性时，

clf.named_steps['classifier'].feature_importances_

我得到一个形状数组(104，)

array([1.39312528e-01, 1.92086014e-01, 1.15276068e-01, 4.01797967e-02,
       7.08805229e-02, 3.99687904e-03, 6.68727677e-03, 0.00000000e+00,
       1.02021005e-02, 5.06637671e-03, 7.97826949e-03, 5.64939616e-03,
       0.00000000e+00, 9.09583016e-04, 1.84022196e-03, 9.29047900e-04,
       1.74001682e-04, 8.55362503e-05, 2.32440522e-03, 4.65023589e-04,
       4.13278579e-03, 3.68265995e-03, 1.78503960e-02, 8.33035943e-03,
       6.94454768e-03, 1.75988171e-02, 5.40933687e-04, 7.51299294e-03,
       6.07480929e-03, 2.28627732e-03, 1.32219786e-03, 1.92990938e-01,
       1.18517448e-03, 1.61377248e-03, 5.72167000e-04, 1.34920904e-03,
       5.41685180e-03, 0.00000000e+00, 9.16416279e-03, 1.05824472e-02,
       3.07744966e-03, 3.07152204e-03, 5.06657379e-03, 5.21819782e-03,
       0.00000000e+00, 7.49534136e-03, 2.83936918e-03, 8.62398812e-03,
       5.78720378e-03, 5.37536831e-03, 2.99744077e-03, 1.87247908e-03,
       4.87696805e-04, 1.58422357e-03, 2.20761597e-03, 5.57396015e-03,
       1.17619435e-03, 1.87465473e-03, 4.08710965e-03, 6.73508851e-04,
       6.02887867e-03, 2.38887308e-03, 4.52029746e-03, 7.28018074e-05,
       5.13158297e-04, 2.66768058e-04, 0.00000000e+00, 3.28378333e-04,
       0.00000000e+00, 8.55362503e-05, 0.00000000e+00, 7.89886262e-04,
       1.84475320e-04, 1.37879652e-03, 0.00000000e+00, 3.27800552e-04,
       1.95189232e-04, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00,
       0.00000000e+00, 9.00792536e-04, 0.00000000e+00, 2.20606426e-04,
       5.82787439e-04, 4.85000896e-04, 5.33409400e-04, 0.00000000e+00,
       8.75840665e-04, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00,
       4.65546160e-04, 3.37472507e-04, 2.50837357e-04, 2.52474592e-04,
       0.00000000e+00, 1.47818105e-04, 3.06829767e-04, 3.73651596e-04,
       1.58778645e-04, 4.40566013e-03, 8.55362503e-05, 2.51672361e-04])

这是不正确的，因为我只有13个功能。我知道原因是OneHotenCoding。

如何获取实际功能重要性？

推荐答案

恐怕您在此无法获得您的初始功能的重要性。您的决策树对它们一无所知；它看到和知道的唯一事情就是编码的那些，其他什么都不知道。

您可能想尝试permutation importance，这比基于树的功能重要性有几个优势；它也很容易适用于管道-请参阅Permutation importance using a Pipeline in SciKit-Learn。

这篇关于如何获得具有预处理和分类步骤的决策树管道的特征重要性？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！