Spark MLLib SVM 输出的分数是什么意思? [英] What does the score of the Spark MLLib SVM output mean?

问题描述

我不明白来自 Spark MLLib 算法的 SVM 分类器的输出.我想将分数转换为概率，以便获得属于某个类的数据点的概率(SVM 在该类上进行训练，也就是多类问题)(另见此线程).目前尚不清楚分数意味着什么.是到超平面的距离吗?我如何从中获得概率?

I do not understand the output of the SVM classifier from the Spark MLLib algorithm. I want to convert the score to a probability, so that I get a probability for a data-point belonging to a certain class (on which the SVM is trained, a.k.a. multi-class problem) (see also this thread). It is unclear what the score means. Is it the distance to the hyperplane? How do I get the probabilities from it?

推荐答案

import org.apache.spark.mllib.classification.{SVMModel, SVMWithSGD}
import org.apache.spark.mllib.evaluation.BinaryClassificationMetrics
import org.apache.spark.mllib.util.MLUtils

// Load training data in LIBSVM format.
val data = MLUtils.loadLibSVMFile(sc, "data/mllib/sample_libsvm_data.txt")

// Split data into training (60%) and test (40%).
val splits = data.randomSplit(Array(0.6, 0.4), seed = 11L)
val training = splits(0).cache()
val test = splits(1)

// Run training algorithm to build the model
val numIterations = 100
val model = SVMWithSGD.train(training, numIterations)

// Clear the default threshold.
model.clearThreshold()

// Compute raw scores on the test set.
val scoreAndLabels = test.map { point =>
  val score = model.predict(point.features)
  (score, point.label)
}

// Get evaluation metrics.
val metrics = new BinaryClassificationMetrics(scoreAndLabels)
val auROC = metrics.areaUnderROC()

println("Area under ROC = " + auROC)

// Save and load model
model.save(sc, "myModelPath")
val sameModel = SVMModel.load(sc, "myModelPath")

如果您在 MLLib 中使用 SVM 模块，它们会为您提供 AUC，即 ROC 曲线下的面积，相当于准确度".希望有帮助.

If you are using SVM module in MLLib , they provide you the AUC which is area under ROC curve and it is equivalent to "Accuracy" . Hope it helps.

这篇关于Spark MLLib SVM 输出的分数是什么意思?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！