如何使用 Spark 在 MySQL (JDBC) 上执行连接? [英] How to perform join on MySQL (JDBC) with Spark?

问题描述

我想通过 Spark 从 MySQL 读取数据.我看到的 API 能够从特定表中读取数据.类似的东西，

I would like to read data from MySQL through Spark. The API which I saw is able to read data from specific table. something like,

val prop = new java.util.Properties
prop.setProperty("user", "<username>")
prop.setProperty("password", "<password>")

sparkSession.read.jdbc("jdbc:mysql://????:3306/???", "some-table", prop)

现在，我想对连接表执行查询.有谁知道怎么做(在数据库方面，而不是使用 Spark SQL)?

Now, I would like to perform a query for join tables. Does anyone know how to do it (on the database side, not with Spark SQL) ?

谢谢，

伊兰

推荐答案

您需要使用table"参数作为查询:

You'll need to use the "table " argument as a query:

val table = "(SELECT foo JOIN bar ON foo.id = bar.id) as t"

spark.read.jdbc("jdbc:mysql://????:3306/???", table, prop)

您应该注意，为您的查询指定别名很重要，否则将不起作用.

You should note that giving an alias to your query is important or this won't work.

这篇关于如何使用 Spark 在 MySQL (JDBC) 上执行连接?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！