'KMeansModel'对象在apache pyspark中没有属性'computeCost' [英] 'KMeansModel' object has no attribute 'computeCost' in apache pyspark

问题描述

我正在pyspark中尝试集群模型.我正在尝试获取适合不同K值的聚类的均方成本

I'm experimenting with a clustering model in pyspark. I'm trying to get the mean squared cost of the cluster fit for different values of K

def meanScore(k,df):
  inputCol = df.columns[:38]
  assembler = VectorAssembler(inputCols=inputCols,outputCol="features")
  kmeans = KMeans().setK(k)
  pipeModel2 = Pipeline(stages=[assembler,kmeans])
  kmeansModel = pipeModel2.fit(df).stages[-1]
  kmeansModel.computeCost(assembler.transform(df))/data.count()

当我尝试调用此函数来计算数据帧中K的不同值的成本时

When I try to call this function to compute costs for different values of K in the dataframe

for k in range(20,100,20):
  sc = meanScore(k,numericOnly)
  print((k,sc))

我收到一个属性错误，因为 AttributeError:'KMeansModel'对象没有属性'computeCost'

I receive an attribute error as AttributeError: 'KMeansModel' object has no attribute 'computeCost'

我对pyspark还是陌生的，只是在学习，我对此表示衷心的感谢.谢谢

I'm fairly new to pyspark and am just learning, I sincerely appreciate any help with this. Thanks

推荐答案

在Spark 3.0.0

It is deprecated in Spark 3.0.0 Docs suggest using the evaluator.

Note Deprecated in 3.0.0. It will be removed in future versions. 
Use ClusteringEvaluator instead. You can also get the cost on the training dataset in the summary.

这篇关于'KMeansModel'对象在apache pyspark中没有属性'computeCost'的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！