tm 包错误“无法将 DocumentTermMatrix 转换为普通矩阵,因为向量太大" [英] tm package error "Cannot convert DocumentTermMatrix into normal matrix since vector is too large"
问题描述
我创建了一个 DocumentTermMatrix,其中包含 1859 个文档(行)和 25722 个(列).为了对该矩阵执行进一步的计算,我需要将其转换为常规矩阵.我想使用 as.matrix()
命令.但是,它返回以下错误:无法分配大小为 364.8 MB 的向量.
I have created a DocumentTermMatrix that contains 1859 documents (rows) and 25722 (columns). In order to perform further calculations on this matrix I need to convert it to a regular matrix. I want to use the as.matrix()
command. However, it returns the following error: cannot allocate vector of size 364.8 MB.
> corp
A corpus with 1859 text documents
> mat<-DocumentTermMatrix(corp)
> dim(mat)
[1] 1859 25722
> is(mat)
[1] "DocumentTermMatrix"
> mat2<-as.matrix(mat)
Fehler: kann Vektor der Größe 364.8 MB nicht allozieren # cannot allocate vector of size 364.8 MB
> object.size(mat)
5502000 bytes
由于某种原因,当对象转换为常规矩阵时,对象的大小似乎会急剧增加.我怎样才能避免这种情况?
For some reason the size of the object seems to increase dramatically whenever it is transformed to a regular matrix. How can I avoid this?
或者是否有其他方法可以在 DocumentTermMatrix 上执行常规矩阵运算?
Or is there an alternative way to perform regular matrix operations on a DocumentTermMatrix?
推荐答案
快速而肮脏的方法是将数据从 Matrix
等外部包导出到稀疏矩阵对象中.
The quick and dirty way is to export your data into a sparse matrix object from an external package like Matrix
.
> attributes(dtm)
$names
[1] "i" "j" "v" "nrow" "ncol" "dimnames"
$class
[1] "DocumentTermMatrix" "simple_triplet_matrix"
$Weighting
[1] "term frequency" "tf"
dtm 对象具有 i、j 和 v 属性,它们是 DocumentTermMatrix 的内部表示.使用:
The dtm object has the i, j and v attributes which is the internal representation of your DocumentTermMatrix. Use:
library("Matrix")
mat <- sparseMatrix(
i=dtm$i,
j=dtm$j,
x=dtm$v,
dims=c(dtm$nrow, dtm$ncol)
)
你就完成了.
对象之间的简单比较:
> mat[1,1:100]
> head(as.vector(dtm[1,]), 100)
每个都会给你完全相同的输出.
will each give you the exact same output.
这篇关于tm 包错误“无法将 DocumentTermMatrix 转换为普通矩阵,因为向量太大"的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!