如何在pyspark中将DenseMatrix转换为spark DataFrame? [英] How to convert DenseMatrix to spark DataFrame in pyspark?

问题描述

除了以下使用 Scala 的示例外，我没有找到任何 pyspark 代码来将矩阵转换为火花数据帧.有谁知道如何改用python?

如何将 mllib 矩阵转换为火花数据框?

解决方案

我们可以使用 toArray() 方法将 DenseMatrix 转换为 numpy ndarray 和 tolist() 转换从数组到列表.

<预><代码>>>>m = DenseMatrix(2, 2, range(4))>>>米DenseMatrix(2, 2, [0.0, 1.0, 2.0, 3.0], False)>>>行 = m.toArray().tolist()>>>行[[0.0, 2.0], [1.0, 3.0]]>>>df = spark.createDataFrame(rows,['col1','col2'])>>>df.show()+----+----+|col1|col2|+----+----+|0.0|2.0||1.0|3.0|+----+----+

I didn't find any pyspark code to convert matrix to spark dataframe except the following example using Scala. Does anyone know how to use python instead?

How to convert a mllib matrix to a spark dataframe?

解决方案

We can use toArray() method to convert DenseMatrix to numpy ndarray and tolist() to convert from array to list.

>>> m = DenseMatrix(2, 2, range(4))
>>> m
DenseMatrix(2, 2, [0.0, 1.0, 2.0, 3.0], False)
>>> rows = m.toArray().tolist()
>>> rows
[[0.0, 2.0], [1.0, 3.0]]
>>> df = spark.createDataFrame(rows,['col1','col2'])
>>> df.show()
+----+----+
|col1|col2|
+----+----+
| 0.0| 2.0|
| 1.0| 3.0|
+----+----+

这篇关于如何在pyspark中将DenseMatrix转换为spark DataFrame?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！