Apache Spark SQL是否支持MERGE子句? [英] Does Apache Spark SQL support MERGE clause?

问题描述

Apache Spark SQL是否支持类似于Oracle的MERGE SQL子句的MERGE子句?

Does Apache Spark SQL support MERGE clause that's similar to Oracle's MERGE SQL clause?

MERGE into <table> using (
  select * from <table1>
    when matched then update...
       DELETE WHERE...
    when not matched then insert...
)

推荐答案

它以Delta Lake作为存储格式:df.write.format("delta").save("/data/events").

It does with Delta Lake as storage format : df.write.format("delta").save("/data/events").

DeltaTable.forPath(spark, "/data/events/")
  .as("events")
  .merge(
    updatesDF.as("updates"),
    "events.eventId = updates.eventId")
  .whenMatched
  .updateExpr(
    Map("data" -> "updates.data"))
  .whenNotMatched
  .insertExpr(
    Map(
      "date" -> "updates.date",
      "eventId" -> "updates.eventId",
      "data" -> "updates.data"))
  .execute()

您还需要delta包:

You also need the delta package:

<dependency>
  <groupId>io.delta</groupId>
  <artifactId>delta-core_2.11</artifactId>
  <version>xxxx</version>
</dependency>

请参见 https://docs.delta.io/0.4.0/delta-update.html 了解更多详情

这篇关于Apache Spark SQL是否支持MERGE子句?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！