如何在没有DataFrames/SparkContext的情况下评估spark.ml模型? [英] How to eval spark.ml model without DataFrames/SparkContext?

问题描述

使用Spark MLLib，我将建立一个模型(如RandomForest)，然后可以通过加载模型并在其上使用predict并传递特征向量来在Spark外部评估它.

With Spark MLLib, I'd build a model (like RandomForest), and then it was possible to eval it outside of Spark by loading the model and using predict on it passing a vector of features.

对于Spark ML来说，predict现在被称为transform，仅作用于DataFrame.

It seems like with Spark ML, predict is now called transform and only acts on a DataFrame.

有什么方法可以在Spark外部构建DataFrame，因为似乎需要一个SparkContext来构建DataFrame?

Is there any way to build a DataFrame outside of Spark since it seems like one needs a SparkContext to build a DataFrame?

我想念什么吗?

推荐答案

回复:有什么方法可以在Spark之外构建DataFrame?

不可能. DataFrames驻留在SQLContext内，SparkContext驻留在其中.也许您可以解决 的问题，但是整个故事是DataFrame和SparkContext之间的连接是设计使然的.

It is not possible. DataFrames live inside SQLContext with it living in SparkContext. Perhaps you could work it around somehow, but the whole story is that the connection between DataFrames and SparkContext is by design.

这篇关于如何在没有DataFrames/SparkContext的情况下评估spark.ml模型?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！