如何将模型结果保存到文本文件? [英] How to save results of Model to text file?

问题描述

我正在尝试将从模型生成的频繁项目集保存到文本文件中.该代码是Spark ML库中FPGrowth示例的示例.

I am trying to save the Frequent itemsets generated from the model to a text file. The code is an example of FPGrowth example in Spark ML library.

直接在模型上使用saveAsTextFile会写入RDD位置，而不是实际值.

Using saveAsTextFile directly on the model writes the RDD locations and not the actual values.

import org.apache.spark.mllib.fpm.FPGrowth
import org.apache.spark.rdd.RDD

val data = sc.textFile("/home/ponny/Freq")
val data1 = sc.textFile("/home/ponny/Scala_Examples/test.txt")
val transactions: RDD[Array[String]] = data.map(s => s.trim.split(' '))
val tdata: RDD[Array[String]] = data1.map(s => s.trim.split(' '))

val fpg = new FPGrowth().setMinSupport(0.0).setNumPartitions(10)
val model = fpg.run(transactions)

model.freqItemsets.collect().foreach { itemset =>
  println(itemset.items.mkString("[", ",", "]") + ", " + itemset.freq)
}
model.freqItemsets.saveAsTextFile("/home/ponny/Freq_op")

文本文件中生成的输出就像

The output generated in text file is like

org.apache.spark.mllib.fpm.FPGrowth$FreqItemset@5b27c9
org.apache.spark.mllib.fpm.FPGrowth$FreqItemset@2a7acd
org.apache.spark.mllib.fpm.FPGrowth$FreqItemset@d4d011
org.apache.spark.mllib.fpm.FPGrowth$FreqItemset@1fd4350

请指导.

推荐答案

与打印值完全相同.首先构建所需的输出字符串:

Exactly the same way how you print the values. Build desired output string first:

model.freqItemsets
  .map { fi => s"""[${fi.items.mkString(",")}], ${fi.freq}""" }
  .saveAsTextFile(path)

这篇关于如何将模型结果保存到文本文件?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！