关于火花矩阵的基本线性代数 [英] Basic linear algebra on spark matrices

问题描述

我正在尝试在存储为火花RowMatrix的矩阵上运行一些基本的线性代数运算(特别是转置，点积和逆)，如此处所述

I am trying to run some basic linear algebra operations (specifically transpose, dot product, and inverse) on a matrix stored as a spark RowMatrix as described herehere (using the Python API). Following the example in the docs (for my case I will have many more rows in the matrix, hence the need for Spark), suppose I have something like this:

from pyspark.mllib.linalg.distributed import RowMatrix
# Create an RDD of vectors.
rows = sc.parallelize([[1, 2, 3], [4, 5, 6], [7, 8, 9], [10, 11, 12]])
# Create a RowMatrix from an RDD of vectors.
mat = RowMatrix(rows)

鉴于这种分布式矩阵，是否存在用于进行矩阵转置和点积的例程，例如:

Given such a distributed matrix, are there existing routines for doing matrix transpose and dot product, e.g:

dot(mat.T,mat)

还是矩阵逆?

inverse(mat)

我似乎在文档中找不到任何有关此的内容.寻找(a)指向相关文档的指针或(b)一种自己实现的方法.

I can't seem to find anything in the documentation about this. Looking for either (a) a pointer to the relevant docs or (b) a method for implementing this myself.

推荐答案

目前(Spark 1.6.0) pyspark.mllib.linalg.distributed API仅限于基本操作，例如计算行数/列和类型之间的转换.

As for now (Spark 1.6.0) pyspark.mllib.linalg.distributed API is limited to basic operations like counting rows/columns and transformations between types.

Scala API支持更广泛的方法集，包括乘法( RowMatrix.multiply ， Indexed.RowMatrix.multiply )，换位，SVD( IndexedRowMatrix.computeSVD )，QR分解( RowMatrix.tallSkinnyQR )，格拉姆矩阵计算( computeGramianMatrix )，PCA( RowMatrix.computePrincipalComponents )可以是用于实现更复杂的线性代数函数.

Scala API supports a broader set of methods including multiplication (RowMatrix.multiply, Indexed.RowMatrix.multiply), transposition, SVD (IndexedRowMatrix.computeSVD), QR decomposition (RowMatrix.tallSkinnyQR), Grammian Matrix computation (computeGramianMatrix), PCA (RowMatrix.computePrincipalComponents) which can be used to implement more complex linear algebra functions.

这篇关于关于火花矩阵的基本线性代数的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！