流畅的代码生成器带有 > 的表22列 [英] Slick codegen & tables with > 22 columns
问题描述
我是 Slick 的新手.我正在使用 Scala、ScalaTest 和 Slick 为 Java 应用程序创建测试套件.我使用 slick 在测试前准备数据并在测试后对数据进行断言.使用的数据库有一些超过 22 列的表.我使用 slick-codegen 来生成我的架构代码.
I'm new to Slick. I'm creating a test suite for a Java application with Scala, ScalaTest and Slick. I'm using slick to prepare data before the test and to do assertions on the data after the test. The database used has some tables with more than 22 columns. I use slick-codegen to generate my schema code.
对于超过 22 列的表，slick-codegen 不会生成案例类，而是基于 HList 的自定义类型和伴随的构造函数"方法.据我了解，这是因为元组和案例类只能有 22 个字段的限制.代码的生成方式，Row对象的字段只能通过索引访问.
For tables with more than 22 columns, slick-codegen does not generate a case class, but a HList-based custom type and a companion ‘constructor’ method. As I understand it, this is because the limitation that tuples and case classes can only have 22 fields. The way the code is generated, the fields of a Row-object can only be accessed by index.
我对此有几个问题:
- 据我所知，Scala 2.11 中已经修复了案例类的 22 个字段限制，对吗?
- 如果是这样，是否可以自定义 slick-codegen 来为所有表生成案例类?我调查了一下:我设法在一个被覆盖的
SourceCodeGenerator中设置了override def hlistEnabled = false.但这导致Cannot generate tuple for >22 列，请设置 hlistEnable=true 或覆盖复合.所以我不明白能够取消 HList.可能问题在于或覆盖复合"部分，但我不明白这是什么意思. - 在 slick 和 22 列上搜索互联网，我遇到了一些基于嵌套元组的解决方案.是否可以自定义代码生成器以使用这种方法?
- 如果生成具有 > 22 个字段的 case 类的代码不是一个可行的选择，我认为可以生成一个普通类，它对每一列都有一个访问器"功能，从而提供一个来自索引的映射"-based 访问基于名称的访问.我很乐意自己实现这一代，但我想我需要一些指示从哪里开始.我认为它应该能够为此覆盖标准代码生成器.对于某些自定义数据类型，我已经使用了重写的
SourceCodeGenerator.但是除了这个用例之外，代码生成器的文档对我的帮助不大.
- For what I understand, the 22 fields restriction for case classes is already fixed in Scala 2.11, right?
- If that's the case, would it be possible to customize slick-codegen to generate case classes for all tables? I looked into this: I managed to set
override def hlistEnabled = falsein an overriddenSourceCodeGenerator. But this results inCannot generate tuple for > 22 columns, please set hlistEnable=true or override compound.So I don’t get the point of being able to disbale HList. May be the catch is in the ‘or override compound’ part, but I don't understand what that means. - Searching the internet on slick and 22 columns, I came across some solutions based on nested tuples. Would it be possible to customize the codegen to use this approach?
- If generating code with case classes with > 22 fields is not a viable option, I think it would be possible to generate an ordinary class, which has an ‘accessor’ function for each column, thus providing a ‘mapping’ from index-based access to name-based access. I’d be happy to implement the generation for this myself, but I think I need some pointers where to start. I think it should be able to override the standard codegen for this. I already use an overridden
SourceCodeGeneratorfor some custom data types. But apart from this use case, the documentation of the code generator does not help me that much.
我真的很感谢这里的帮助.提前致谢！
I would really appreciate some help here. Thanks in advance!
推荐答案
我最终进一步定制了 slick-codegen.首先，我会回答我自己的问题，然后我会发布我的解决方案.
I ended up further customizing slick-codegen. First, I'll answer my own questions, then I'll post my solution.
- 案例类可能会取消 22 个元数限制，但不适用于元组.并且 slick-codegen 还生成了一些元组，当我问这个问题时我并没有完全意识到.
- 不相关，请参阅答案 1.(如果元组的 22 个元数限制也被取消，这可能会变得相关.)
- 我选择不进一步调查这个问题，所以这个问题暂时没有答案.
- 这就是我最终采用的方法.
解决方案:生成的代码
因此，我最终为超过 22 列的表生成了普通"类.让我举例说明我现在生成的内容.(生成器代码如下.)(出于简洁和可读性原因，此示例少于 22 列.)
Solution: the generated code
So, I ended up generating "ordinary" classes for tables with more than 22 columns. Let me give an example of what I generate now. (Generator code follows below.) (This example has less than 22 columns, for brevity and readability reasons.)
case class BigAssTableRow(val id: Long, val name: String, val age: Option[Int] = None)
type BigAssTableRowList = HCons[Long,HCons[String,HCons[Option[Int]]], HNil]
object BigAssTableRow {
def apply(hList: BigAssTableRowList) = new BigAssTableRow(hlist.head, hList.tail.head, hList.tail.tail.head)
def unapply(row: BigAssTableRow) = Some(row.id :: row.name :: row.age)
}
implicit def GetResultBoekingenRow(implicit e0: GR[Long], e1: GR[String], e2: GR[Optional[Int]]) = GR{
prs => import prs._
BigAssTableRow.apply(<<[Long] :: <<[String] :: <<?[Int] :: HNil)
}
class BigAssTable(_tableTag: Tag) extends Table[BigAssTableRow](_tableTag, "big_ass") {
def * = id :: name :: age :: :: HNil <> (BigAssTableRow.apply, BigAssTableRow.unapply)
val id: Rep[Long] = column[Long]("id", O.PrimaryKey)
val name: Rep[String] = column[String]("name", O.Length(255,varying=true))
val age: Rep[Option[Int]] = column[Option[Int]]("age", O.Default(None))
}
lazy val BigAssTable = new TableQuery(tag => new BigAssTable(tag))
最难的部分是找出 * 映射在 Slick 中的工作原理.没有太多文档，但我找到了 this Stackoverflow answer有启发.
The hardest part was finding out how the * mapping works in Slick. There's not much documentation, but I found this Stackoverflow answer rather enlightening.
我创建了 BigAssTableRow object 以使 HList 的使用对客户端代码透明.请注意，对象中的 apply 函数重载了 case 类中的 apply.所以我仍然可以通过调用BigAssTableRow(id: 1L, name: "Foo")来创建实体，而*投影仍然可以使用apply 采用 HList 的函数.
I created the BigAssTableRow object to make the use of HList transparent for the client code. Note that the apply function in the object overloads the apply from the case class. So I can still create entities by calling BigAssTableRow(id: 1L, name: "Foo"), while the * projection can still use the apply function that takes an HList.
所以，我现在可以做这样的事情:
So, I can now do things like this:
// I left out the driver import as well as the scala.concurrent imports
// for the Execution context.
val collection = TableQuery[BigAssTable]
val row = BigAssTableRow(id: 1L, name: "Qwerty") // Note that I leave out the optional age
Await.result(db.run(collection += row), Duration.Inf)
Await.result(db.run(collection.filter(_.id === 1L).result), Duration.Inf)
对于此代码，无论在后台使用元组还是 HList，它都是完全透明的.
For this code it's totally transparent wether tuples or HLists are used under the hood.
我会在这里发布我的整个生成器代码.它并不完美；如果您有改进建议，请告诉我！巨大的部分只是从 slick.codegen.AbstractSourceCodeGenerator 和相关类中复制，然后稍作更改.还有一些和这个问题没有直接关系的东西，比如java.time.*数据类型的添加和特定表的过滤.我把它们留在了，因为它们可能有用.另请注意，此示例适用于 Postgres 数据库.
I'll just post my entire generator code here. It's not perfect; please let me know if you have suggestions for improvement! Huge parts are just copied from the slick.codegen.AbstractSourceCodeGenerator and related classes and then slightly changed. There are also some things that are not directly related to this question, such as the addition of the java.time.* data types and the filtering of specific tables. I left them in, because they might be of use. Also note that this example is for a Postgres database.
import slick.codegen.SourceCodeGenerator
import slick.driver.{JdbcProfile, PostgresDriver}
import slick.jdbc.meta.MTable
import slick.model.Column
import scala.concurrent.Await
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.duration.Duration
object MySlickCodeGenerator {
val slickDriver = "slick.driver.PostgresDriver"
val jdbcDriver = "org.postgresql.Driver"
val url = "jdbc:postgresql://localhost:5432/dbname"
val outputFolder = "/path/to/project/src/test/scala"
val pkg = "my.package"
val user = "user"
val password = "password"
val driver: JdbcProfile = Class.forName(slickDriver + "$").getField("MODULE$").get(null).asInstanceOf[JdbcProfile]
val dbFactory = driver.api.Database
val db = dbFactory.forURL(url, driver = jdbcDriver, user = user, password = password, keepAliveConnection = true)
// The schema is generated using Liquibase, which creates these tables that I don't want to use
def excludedTables = Array("databasechangelog", "databasechangeloglock")
def tableFilter(table: MTable): Boolean = {
!excludedTables.contains(table.name.name) && schemaFilter(table.name.schema)
}
// There's also an 'audit' schema in the database, I don't want to use that one
def schemaFilter(schema: Option[String]): Boolean = {
schema match {
case Some("public") => true
case None => true
case _ => false
}
}
// Fetch data model
val modelAction = PostgresDriver.defaultTables
.map(_.filter(tableFilter))
.flatMap(PostgresDriver.createModelBuilder(_, ignoreInvalidDefaults = false).buildModel)
val modelFuture = db.run(modelAction)
// customize code generator
val codegenFuture = modelFuture.map(model => new SourceCodeGenerator(model) {
// add custom import for added data types
override def code = "import my.package.Java8DateTypes._" + "\n" + super.code
override def Table = new Table(_) {
table =>
// Use different factory and extractor functions for tables with > 22 columns
override def factory = if(columns.size == 1) TableClass.elementType else if(columns.size <= 22) s"${TableClass.elementType}.tupled" else s"${EntityType.name}.apply"
override def extractor = if(columns.size <= 22) s"${TableClass.elementType}.unapply" else s"${EntityType.name}.unapply"
override def EntityType = new EntityTypeDef {
override def code = {
val args = columns.map(c =>
c.default.map( v =>
s"${c.name}: ${c.exposedType} = $v"
).getOrElse(
s"${c.name}: ${c.exposedType}"
)
)
val callArgs = columns.map(c => s"${c.name}")
val types = columns.map(c => c.exposedType)
if(classEnabled){
val prns = (parents.take(1).map(" extends "+_) ++ parents.drop(1).map(" with "+_)).mkString("")
s"""case class $name(${args.mkString(", ")})$prns"""
} else {
s"""
/** Constructor for $name providing default values if available in the database schema. */
case class $name(${args.map(arg => {s"val $arg"}).mkString(", ")})
type ${name}List = ${compoundType(types)}
object $name {
def apply(hList: ${name}List): $name = new $name(${callArgs.zipWithIndex.map(pair => s"hList${tails(pair._2)}.head").mkString(", ")})
def unapply(row: $name) = Some(${compoundValue(callArgs.map(a => s"row.$a"))})
}
""".trim
}
}
}
override def PlainSqlMapper = new PlainSqlMapperDef {
override def code = {
val positional = compoundValue(columnsPositional.map(c => if (c.fakeNullable || c.model.nullable) s"<<?[${c.rawType}]" else s"<<[${c.rawType}]"))
val dependencies = columns.map(_.exposedType).distinct.zipWithIndex.map{ case (t,i) => s"""e$i: GR[$t]"""}.mkString(", ")
val rearranged = compoundValue(desiredColumnOrder.map(i => if(columns.size > 22) s"r($i)" else tuple(i)))
def result(args: String) = s"$factory($args)"
val body =
if(autoIncLastAsOption && columns.size > 1){
s"""
val r = $positional
import r._
${result(rearranged)} // putting AutoInc last
""".trim
} else {
result(positional)
}
s"""
implicit def $name(implicit $dependencies): GR[${TableClass.elementType}] = GR{
prs => import prs._
${indent(body)}
}
""".trim
}
}
override def TableClass = new TableClassDef {
override def star = {
val struct = compoundValue(columns.map(c=>if(c.fakeNullable)s"Rep.Some(${c.name})" else s"${c.name}"))
val rhs = s"$struct <> ($factory, $extractor)"
s"def * = $rhs"
}
}
def tails(n: Int) = {
List.fill(n)(".tail").mkString("")
}
// override column generator to add additional types
override def Column = new Column(_) {
override def rawType = {
typeMapper(model).getOrElse(super.rawType)
}
}
}
})
def typeMapper(column: Column): Option[String] = {
column.tpe match {
case "java.sql.Date" => Some("java.time.LocalDate")
case "java.sql.Timestamp" => Some("java.time.LocalDateTime")
case _ => None
}
}
def doCodeGen() = {
def generator = Await.result(codegenFuture, Duration.Inf)
generator.writeToFile(slickDriver, outputFolder, pkg, "Tables", "Tables.scala")
}
def main(args: Array[String]) {
doCodeGen()
db.close()
}
}
这篇关于流畅的代码生成器带有 > 的表22列的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
