在Spark中将CategoricalFeaturesInfo与DecisionTreeClassifier方法一起使用 [英] Using CategoricalFeaturesInfo with DecisionTreeClassifier method in Spark

问题描述

我必须使用以下代码:

val dt = new DecisionTreeClassifier().setLabelCol("indexedLabel").setFeaturesCol("indexedFeatures").setImpurity(impurity).setMaxBins(maxBins).setMaxDepth(maxDepth);

我需要添加分类特征信息，以便决策树不会将indexedCategoricalFeatures视为数字.我有这张地图:

I need to add categorical features information so that the decision tree doesn't treat the indexedCategoricalFeatures as numerical. I have this map:

val categoricalFeaturesInfo = Map(143 -> 126, 144 -> 5, 145 -> 216, 146 -> 100, 147 -> 14, 148 -> 8, 149 -> 19, 150 -> 7);

但是，它仅与DecisionTree.trainClassifier方法一起使用.我无法使用此方法，因为它接受的参数与我所接受的参数不同...我真的希望能够使用具有正确处理的分类特征的DecisionTreeClassifie r.

However it only works with DecisionTree.trainClassifier method. I can't use this method because it accepts different arguments than the one I have... I would really want to be able to use the DecisionTreeClassifier with categorical features treated properly.

感谢您的帮助！

推荐答案

您正在混合使用不同方法来分类数据的两个不同的API:

You're mixing two different APIs which take different approach to categorical data:

基于
• RDD的o.a.s.mllib，它通过传递categoricalFeaturesInfo映射提供了所需的元数据.
• Dataset(DataFrame)o.a.s.ml，它使用列元数据来确定变量类型.如果您正确使用ML转换器创建功能，则应该为您自动处理，否则，您将必须手动提供元数据.

• RDD based o.a.s.mllib which provides required metadata by passing categoricalFeaturesInfo map.
• Dataset (DataFrame) o.a.s.ml which is using column metadata to determine variable types. If you correctly use ML transformers to create features this should be handled automatically for you, otherwise you'll have to provide metadata manually.

这篇关于在Spark中将CategoricalFeaturesInfo与DecisionTreeClassifier方法一起使用的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！