节能与DecisionTreeModel星火ML管道 [英] Saving a Pipeline with DecisionTreeModel Spark ML

问题描述

上下文：

我有一个星火ML管道包含VectorAssembler，StringIndexer和DecisionTreeClassifier。使用这条管道，我能够成功地适应模型和转换我的数据帧。我想用来存储这种模式以供日后使用，但我不断收到以下错误：

I have a Spark ML pipeline that contains a VectorAssembler, StringIndexer, and a DecisionTreeClassifier. Using this pipeline I am able to successfully fit the model and transform my data frame. I would like to store this model for future use, but I keep getting the following error:

Pipeline write will fail on this Pipeline because it contains a stage which does not implement Writable. 
Non-Writable stage: dtc_9c04161ed2d1 of type class org.apache.spark.ml.classification.DecisionTreeClassificationModel

我试图

val pipeline = new Pipeline().setStages(Array(assembler, labelIndexer, dt))
val model = pipeline.fit(dfIndexed)
model.write.overwrite().save("test/model/pipeline")

这正常工作，当我删除分类（即DT）。是否有保存DecisionTreeClassifier模型的方法吗？

This works properly when I remove the classifier (i.e. dt). Is there a way of saving a DecisionTreeClassifier model?

我的数据包括，我必须映射回其原始形态一些索引分类值的（我知道这将需要使用IndexToString）。我使用的Spark 1.6。

My data consists of some indexed categorical values that I must map back to their original form (I know this will require using IndexToString). I am using Spark 1.6.

推荐答案

这不能做星火1.6。这个问题正在跟踪这里。

This cannot be done as of Spark 1.6. The issue is being tracked here.

这篇关于节能与DecisionTreeModel星火ML管道的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！