Apache Spark 移动平均线 [英] Apache Spark Moving Average

问题描述

我在 HDFS 中有一个包含时间序列数据点(雅虎股票价格)的大文件.

I have a huge file in HDFS having Time Series data points (Yahoo Stock prices).

我想找到时间序列的移动平均值，我该如何编写 Apache Spark 作业来做到这一点.

I want to find the moving average of the Time Series how do I go about writing the Apache Spark job to do that .

推荐答案

您可以使用 MLLIB 中的滑动函数，它可能与 Daniel 的回答做同样的事情.在使用滑动功能之前，您必须按时间对数据进行排序.

You can use the sliding function from MLLIB which probably does the same thing as Daniel's answer. You will have to sort the data by time before using the sliding function.

import org.apache.spark.mllib.rdd.RDDFunctions._

sc.parallelize(1 to 100, 10)
  .sliding(3)
  .map(curSlice => (curSlice.sum / curSlice.size))
  .collect()

这篇关于Apache Spark 移动平均线的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！