在 Spark 中使用 CategoricalFeaturesInfo 和 DecisionTreeClassifier 方法 [英] Using CategoricalFeaturesInfo with DecisionTreeClassifier method in Spark

问题描述

我必须使用此代码:

val dt = new DecisionTreeClassifier().setLabelCol("indexedLabel").setFeaturesCol("indexedFeatures").setImpurity(impurity).setMaxBins(maxBins).setMaxDepth(maxDepth);

我需要添加分类特征信息，以便决策树不会将 indexedCategoricalFeatures 视为数字.我有这张地图:

I need to add categorical features information so that the decision tree doesn't treat the indexedCategoricalFeatures as numerical. I have this map:

val categoricalFeaturesInfo = Map(143 -> 126, 144 -> 5, 145 -> 216, 146 -> 100, 147 -> 14, 148 -> 8, 149 -> 19, 150 -> 7);

但是它只适用于 DecisionTree.trainClassifier 方法.我无法使用此方法，因为它接受的参数与我所拥有的参数不同......我真的希望能够使用 DecisionTreeClassifier 并正确处理分类特征.

However it only works with DecisionTree.trainClassifier method. I can't use this method because it accepts different arguments than the one I have... I would really want to be able to use the DecisionTreeClassifier with categorical features treated properly.

感谢您的帮助！

推荐答案

您正在混合两种不同的 API，它们对分类数据采取不同的方法:

You're mixing two different APIs which take different approach to categorical data:

• RDD 基于 o.a.s.mllib，它通过传递 categoricalFeaturesInfo 映射提供所需的元数据.
• Dataset (DataFrame) o.a.s.ml 使用列元数据来确定变量类型.如果您正确使用 ML 转换器来创建功能，这应该为您自动处理，否则您必须手动提供元数据.

• RDD based o.a.s.mllib which provides required metadata by passing categoricalFeaturesInfo map.
• Dataset (DataFrame) o.a.s.ml which is using column metadata to determine variable types. If you correctly use ML transformers to create features this should be handled automatically for you, otherwise you'll have to provide metadata manually.

这篇关于在 Spark 中使用 CategoricalFeaturesInfo 和 DecisionTreeClassifier 方法的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！