序列化带有或不带有@transient的惰性val时的区别 [英] Difference when serializing a lazy val with or without @transient

问题描述

在处理spark时，有时我需要在每个任务中发送一个不可序列化的对象.

Working on spark, sometimes I need to send a non-serializable object in each task.

常见的模式是@transient lazy val，例如

class A(val a: Int)

def compute(rdd: RDD[Int]) = {
  // lazy val instance = {
  @transient lazy val instance = {
    println("in lazy object")
    new A(1)
  }
  val res = rdd.map(instance.a + _).count()
  println(res)
}

compute(sc.makeRDD(1 to 100, 8))

我发现这里不需要@transient. lazy val可以在执行每个任务时创建不可序列化的内容.但是人们建议使用@transient.

I found that @transient is not necessary here. lazy val can already create the non-serializable upon each task is executed. But people suggest using @transient.

1. 如果序列化时在未初始化的lazy val上设置@transient有什么好处?

1. What is the advantage, if we set @transient on the non-initialized lazy val when serializing it ?

知道没有要序列化的东西，就像上面的例子一样，使未初始化的val瞬态序列化是否有意义?

Does it make sense to make a non-initialized val transient for serialization, knowing that nothing will be serialized, just like in the example above ?

@transient lazy val如何序列化?是将其视为方法还是其他?

How is a @transient lazy val serialized ? Is it treated as a method or something else ?

有关序列化@transient lazy val的一些详细信息，并且已编译的Java字节码很棒.

Some details on serializing @transient lazy val and the compiled java bytecode is awesome.

推荐答案

请参阅此处-在Scala中，lazy val表示一个字段，只有在首次访问该字段时才会计算该字段，然后将其存储以供将来参考.另一方面，使用@transient可以表示一个不应序列化的字段.

In Scala lazy val denotes a field that will only be calculated once it is accessed for the first time and is then stored for future reference. With @transient on the other hand one can denote a field that shall not be serialized.

这篇关于序列化带有或不带有@transient的惰性val时的区别的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！