证明runtimeClass满足Scala中的Bound类型 [英] Prove that a runtimeClass satisfies a type Bound in Scala

问题描述

我有一个方法可以以Parquet形式编写我的一个类 Foo (定义为Thrift).

I have a method that writes one of my classes Foo, which is defined as Thrift, in Parquet form.

  import Foo
  import org.apache.spark.rdd.RDD
  import org.apache.thrift.TBase
  import org.apache.hadoop.mapreduce.Job
  import org.apache.parquet.hadoop.ParquetOutputFormat
  import org.apache.parquet.hadoop.thrift.ParquetThriftOutputFormat

  def writeThriftParquet(rdd: RDD[Foo], outputPath: String): Unit = {
    val job = Job.getInstance()
    ParquetThriftOutputFormat.setThriftClass(job, classOf[Foo])
    ParquetOutputFormat.setWriteSupportClass(job, classOf[Foo])

    rdd
      .map(x => (null, x))
      .saveAsNewAPIHadoopFile(
        outputPath,
        classOf[Void],
        classOf[Foo],
        classOf[ParquetThriftOutputFormat[Foo]],
        job.getConfiguration)
  }

这很好用，但是我更愿意写一个更通用的方法.我尝试了(相对)简单的方法:

This works fine, but I'd prefer to write a more generic method. I tried the (relatively) simple:

  def writeThriftParquetGeneral[A <: TBase[_, _]](rdd: RDD[A], outputPath: String): Unit = {
    val job = Job.getInstance()
    ParquetThriftOutputFormat.setThriftClass(job, classOf[A])
    ParquetOutputFormat.setWriteSupportClass(job, classOf[A])

    rdd
      .map(x => (null, x))
      .saveAsNewAPIHadoopFile(
        outputPath,
        classOf[Void],
        classOf[A],
        classOf[ParquetThriftOutputFormat[A]],
        job.getConfiguration)
  }

但是失败，并显示以下错误:

but that fails with errors like:

 class type required but A found ParquetThriftOutputFormat.setThriftClass(job, classOf[A])
 class type required but A found ParquetOutputFormat.setWriteSupportClass(job, classOf[A])

为了解决这个问题，我使用了 ClassTag ，但是还没有编译的东西.

To try to remedy that, I've used a ClassTag, but haven't gotten things to compile.

  import scala.reflect._
  implicit val ct = ClassTag[Foo](classOf[Foo])

  def writeThriftParquetGeneral[A <: TBase[_, _]](rdd: RDD[A], outputPath: String)(
    implicit tag: ClassTag[A]): Unit = {
    val job = Job.getInstance()

    // The problem line
    ParquetThriftOutputFormat.setThriftClass(job, tag.runtimeClass)

    // Seems OK from here
    ParquetOutputFormat.setWriteSupportClass(job, tag.runtimeClass)

    rdd
      .map(x => (null, x))
      .saveAsNewAPIHadoopFile(
        outputPath,
        classOf[Void],
        tag.runtimeClass,
        classOf[ParquetThriftOutputFormat[A]],
        job.getConfiguration)
  }

此操作在以下行失败: ParquetThriftOutputFormat.setThriftClass(job，tag.runtimeClass)

This fails at the line: ParquetThriftOutputFormat.setThriftClass(job, tag.runtimeClass)

[error]  found   : Class[_$1] where type _$1
[error]  required: Class[_ <: org.apache.thrift.TBase[_, _]]

我很惊讶编译器(Scala 2.11)没有意识到 tag.runtimeClass 必须是 classOf [A] 和 A 满足定义所定义的类型.

I'm surprised the compiler (Scala 2.11) isn't recognizing that tag.runtimeClass must be a classOf[A], and A satisfies the type bound by definition.

推荐答案

ClassTag#runtimeClass 仅返回 Class [_]

https://github.com/scala/scala/blob/2.13.x/src/library/scala/reflect/ClassTag.scala#L55

Class [_< ;: TBase [_，_]] 是不同于 Class [_] (实际上是其子类型)的存在类型

Class[_ <: TBase[_, _]] is an existential type different from Class[_] (actually its subtype)

implicitly[Class[_ <: TBase[_, _]] <:< Class[_]]

尝试将问题行替换为

ParquetThriftOutputFormat.setThriftClass(job, classTag.runtimeClass.asSubclass(classOf[TBase[_, _]]))

这篇关于证明runtimeClass满足Scala中的Bound类型的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！