节能与DecisionTreeModel星火ML管道 [英] Saving a Pipeline with DecisionTreeModel Spark ML
问题描述
上下文:
我有一个星火ML管道包含VectorAssembler,StringIndexer和DecisionTreeClassifier。使用这条管道,我能够成功地适应模型和转换我的数据帧。我想用来存储这种模式以供日后使用,但我不断收到以下错误:
I have a Spark ML pipeline that contains a VectorAssembler, StringIndexer, and a DecisionTreeClassifier. Using this pipeline I am able to successfully fit the model and transform my data frame. I would like to store this model for future use, but I keep getting the following error:
Pipeline write will fail on this Pipeline because it contains a stage which does not implement Writable.
Non-Writable stage: dtc_9c04161ed2d1 of type class org.apache.spark.ml.classification.DecisionTreeClassificationModel
我试图
val pipeline = new Pipeline().setStages(Array(assembler, labelIndexer, dt))
val model = pipeline.fit(dfIndexed)
model.write.overwrite().save("test/model/pipeline")
这正常工作,当我删除分类(即DT)。是否有保存DecisionTreeClassifier模型的方法吗?
This works properly when I remove the classifier (i.e. dt). Is there a way of saving a DecisionTreeClassifier model?
我的数据包括,我必须映射回其原始形态一些索引分类值的(我知道这将需要使用IndexToString)。我使用的Spark 1.6。
My data consists of some indexed categorical values that I must map back to their original form (I know this will require using IndexToString). I am using Spark 1.6.
推荐答案
这不能做星火1.6。这个问题正在跟踪这里。
This cannot be done as of Spark 1.6. The issue is being tracked here.
这篇关于节能与DecisionTreeModel星火ML管道的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!