在Mahout 0.8中运行CVB [英] Run cvb in mahout 0.8

问题描述

当前的Mahout 0.8-SNAPSHOT包括用于主题建模的折叠变型贝叶斯(cvb)版本，并删除了潜在Dirichlet分析(lda)方法，因为可以更好地并行化cvb.不幸的是，只有 lda 的文档，说明了如何运行示例并生成有意义的输出.

The current Mahout 0.8-SNAPSHOT includes a Collapsed Variational Bayes (cvb) version for Topic Modeling and removed the Latent Dirichlet Analysis (lda) approach, because cvb can be parallelized way better. Unfortunately there is only documentation for lda on how to run an example and generate meaningful output.

因此，我想:

• 正确预处理一些文本
• 运行cvb的cvb0_local版本
• 通过查看每个生成的主题中的前n个词来检查结果

推荐答案

所以这是我必须在linux shell中调用的后续Mahout命令. $ MAHOUT_HOME指向我的mahout/bin文件夹.

So here are the subsequent Mahout commands I had to call in a linux shell to do it. $MAHOUT_HOME points to my mahout/bin folder.

$MAHOUT_HOME/mahout seqdirectory \
    -i path/to/directory/with/texts \
    -o out/sequenced

$MAHOUT_HOME/mahout seq2sparse -i out/sequenced \
    -o out/sparseVectors \
    --namedVector \
    -wt tf

$MAHOUT_HOME/mahout rowid \
    -i out/sparseVectors/tf-vectors/ \
    -o out/matrix

$MAHOUT_HOME/mahout cvb0_local \
    -i out/matrix/matrix \
    -d out/sparseVectors/dictionary.file-0 \
    -a 0.5 \
    -top 4 -do out/cvb/do_out \
    -to out/cvb/to_out

通过显示每个主题的前10个词来检查输出:

Inspect the output by showing the top 10 words of each topic:

$MAHOUT_HOME/mahout vectordump \
    -i out/cvb/to_out \
    --dictionary out/sparseVectors/dictionary.file-0 \
    --dictionaryType sequencefile \
    --vectorSize 10 \
    -sort out/cvb/to_out

这篇关于在Mahout 0.8中运行CVB的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！