更改属于RDD的对象中的属性 [英] Changing an attribute in an object that belongs to RDD

问题描述

我有以下代码：

  def generateStoriesnew（outputPath：String，groupedRDD：RDD [（String，Iterable [String ]），isInChurnMode：Boolean，isInChurnPeriod：Boolean）{
 val windowedRDD = groupedRDD.map（SOME CODE）
 var windowedRDD2 = windowedRDD.filter（r => r！= null）.map a => a.churnPeriod（isInChurnPeriod，isInChurnMode））
 val prettyStringRDD = windowedRDD2.map（r => {
 r.toString 
}）
 prettyStringRDD.saveAsTextFile outputPath）
  

}

ChurnPriod函数的代码：

  def churnPeriod（churnPeriod：Boolean，churnMode：Boolean）：Unit = {
 if （churnMode&& rootEventType.equalsIgnoreCase（c））{
 var churnCustStory：CustStoryN = null 
 var nonChurnCustStory：CustStoryN = null 
 var churnPeriodEventStory：mutable.MutableList [StoryEventN] = null 
 var NonChurnEventstory：mutable.MutableList [StoryEventN] = null 
 churnPeriodEventStory = new mutable.MutableList [StoryEventN] 
 NonChurnEventstory = new mutable.MutableList [StoryEventN] 
 var lastEventChurnPeriod = true 
 var currentEventStory = eventStory 
 var max = currentEventStory.length 
 println（max）; 
 if（currentEventStory.size> 0）{
 for（i<  -  0 until currentEventStory.length）{
 var currentEvent = currentEventStory（i）
 if（currentEvent。 timeSenseRootEvent <90）{
 churnPeriodEventStory。+ =（currentEvent）
 // lastEventChurnPeriod = true 
} 
 else {
 NonChurnEventstory。+ = $ b lastEventChurnPeriod = false 
} 
} 
} 
 if（churnPeriod）
 eventStory = churnPeriodEventStory 
 else 
 eventStory = null 
  

}
}

函数不更改属于custstory类的成员的eventstory。我在这里缺少什么？

  class CustStoryN（val custId：String，
 var rootEventType：String，
 var rootEventTime：Long，
 var eventStory：mutable.MutableList [StoryEventN]）
  

my假设是：
1.map不是正确的转换函数，我有
2.churnPeriod函数永远不会调用
3.我不能更改eventstory是cust的成员故事类
有没有人知道如何解决这个问题？

解决方案

调试。只要放几个断点，你可以看到程序是否停在函数内部，以及发生了什么转换。

我的猜测是问题是这行： code> if（currentEventStory.size> 0），因此，以大小0开始的列表永远保持为大小0。另一个选项是 churnPeriod 从不为true，因此，您计算很多，但从不分配到 eventStory 变量。 / p>

您的代码需要一个很好的清理; - ）

I have the following code :

      def generateStoriesnew(outputPath: String, groupedRDD:RDD[(String,Iterable[String])], isInChurnMode: Boolean, isInChurnPeriod: Boolean) {
val windowedRDD = groupedRDD.map(SOME CODE)
 var  windowedRDD2 = windowedRDD.filter(r => r != null).map(a=>a.churnPeriod(isInChurnPeriod,isInChurnMode))  
val prettyStringRDD = windowedRDD2.map(r => {
  r.toString
})
prettyStringRDD.saveAsTextFile(outputPath)

}

and here is the code for ChurnPriod function:

def churnPeriod( churnPeriod:Boolean, churnMode: Boolean): Unit = {  
if (churnMode && rootEventType.equalsIgnoreCase("c")){
  var churnCustStory: CustStoryN = null
  var nonChurnCustStory: CustStoryN = null
  var churnPeriodEventStory: mutable.MutableList[StoryEventN] = null
  var NonChurnEventstory: mutable.MutableList[StoryEventN] = null
  churnPeriodEventStory = new mutable.MutableList[StoryEventN]
  NonChurnEventstory = new mutable.MutableList[StoryEventN]     
  var lastEventChurnPeriod = true
  var currentEventStory = eventStory
  var max = currentEventStory.length
  println(max);
  if (currentEventStory.size > 0) {
    for (i <- 0 until currentEventStory.length) {
      var currentEvent = currentEventStory(i)
      if (currentEvent.timeSenseRootEvent < 90) {          
        churnPeriodEventStory.+=(currentEvent)
        //lastEventChurnPeriod = true
      }
      else {
        NonChurnEventstory.+=(currentEvent)
        lastEventChurnPeriod = false
      }
    }
  }
if (churnPeriod)
  eventStory = churnPeriodEventStory
else
  eventStory=null

} }

but churn period function does not change eventstory which is a member of a custstory class. what am I missing here ?

    class CustStoryN (val custId:String,
             var rootEventType:String,
             var rootEventTime:Long,
             var eventStory:mutable.MutableList[StoryEventN])

my hypothesis is either: 1.map is not the right transformation for the function that I have 2.churnPeriod function never get called 3.I can not change eventstory which is a member of cust story class Does anyone have any idea how I can troubleshoot this problem?

解决方案

This would be trivial to determine by debugging. Just put a few breakpoints and you can see if the program stops inside the function, and what transformations do occur.

My guess would be that the problem is this line: if (currentEventStory.size > 0), thus, a list that starts at size 0 remains at size 0 forever. Another option is that churnPeriod is never true, thus, you compute a lot but never assign to the eventStory variable.

Your code does need a good cleanup ;-)

这篇关于更改属于RDD的对象中的属性的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！