如何将双精度数组的数据帧转换为向量? [英] How to convert a dataframe of array of doubles to Vectors?

问题描述

我有以下带有双精度数组的数据帧，需要将其转换为向量才能将其传递给 ML 算法.有人能帮我解决这个问题吗?

I have the following dataframe with array of doubles that need to be converted to Vectors in order to pass it to an ML algorithm. Can anyone help me with this?

fList: org.apache.spark.sql.DataFrame = [features: array<double>]
+--------------------------------------------------------------------------------+
|features                                                                        |
+--------------------------------------------------------------------------------+
|[2.5046410000000003, 2.1487149999999997, 1.0884870000000002, 3.5877090000000003]|
|[0.9558040000000001, 0.9843780000000002, 0.545025, 0.9979860000000002]          |
+--------------------------------------------------------------------------------+

预期输出:应该看起来像这样.

Expected Output: Should look something like this.

fList: org.apache.spark.sql.DataFrame = [features: vector]

推荐答案

我建议你写一个udf函数

import org.apache.spark.sql.functions._
import org.apache.spark.mllib.linalg.Vectors
def convertArrayToVector = udf((features: mutable.WrappedArray[Double]) => Vectors.dense(features.toArray))

并在 withColumn api

scala> df.withColumn("features", convertArrayToVector($"features"))
res1: org.apache.spark.sql.DataFrame = [features: vector]

希望回答对你有帮助

这篇关于如何将双精度数组的数据帧转换为向量?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！