我们是否可以更新 spark-ml/spark-mllib 中的现有模型? [英] Whether we can update existing model in spark-ml/spark-mllib?

问题描述

我们正在使用 spark-ml 从现有数据构建模型.每天都有新数据出现.

We are using spark-ml to build the model from existing data. New data comes on daily basis.

有没有办法只读取新数据并更新现有模型，而不必每次都读取所有数据并重新训练?

Is there a way that we can only read the new data and update the existing model without having to read all the data and retrain every time?

推荐答案

这取决于您使用的模型，但对于某些 Spark 完全按照您的要求 想要.你可以看看 StreamingKMeans, StreamingLinearRegressionWithSGD, StreamingLogisticRegressionWithSGD 和更广泛的 StreamingLinearAlgorithm.

It depends on the model you're using but for some Spark does exactly what you want. You can look at StreamingKMeans, StreamingLinearRegressionWithSGD, StreamingLogisticRegressionWithSGD and more broadly StreamingLinearAlgorithm.

这篇关于我们是否可以更新 spark-ml/spark-mllib 中的现有模型?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！