Spark MLlib ALS 中的非整数 ID [英] Non-integer ids in Spark MLlib ALS

问题描述

我想用

val ratings = data.map(_.split(',') match {
      case Array(user,item,rate)
      =>
        Rating(user.toInt,item.toInt,rate.toFloat)
    })
val model =  ALS.train(ratings,rank,numIterations,alpha)

但是，我得到的用户数据存储为 Long.当切换到 int 时，它可能会产生错误.我该怎么做才能解决问题?

However, the user data i get are stored as Long. When switched to int, it may produce error. How can i do to solve the problem?

推荐答案

您可以使用支持 Long 标签的 ML 实现之一.RDD 版本与其他实现相比，它的用户友好性明显降低:

You can use one of ML implementations which support Long labels. RDD version it is significantly less user friendly compared to other implementations:

import org.apache.spark.ml.recommendation.ALS
import org.apache.spark.ml.recommendation.ALS.Rating

val ratings = sc.parallelize(Seq(Rating(1L, 2L, 3.0f), Rating(2L, 3L, 5.0f)))

val (userFactors, itemFactors) = ALS.train(ratings)

并且只返回因子但 DataFrame 版本返回一个模型:

and returns only factors but DataFrame version returns a model:

val ratingsDF= ratings.toDF

val alsModel = new ALS().fit(ratingsDF)

这篇关于Spark MLlib ALS 中的非整数 ID的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！