如何实现Spark sql分页查询 [英] how to implement spark sql pagination query

问题描述

有人在Spark sql查询中执行分页吗?

Does anyone how to do pagination in spark sql query?

我需要使用spark sql，但不知道如何进行分页.

I need to use spark sql but don't know how to do pagination.

尝试:

select * from person limit 10, 10

推荐答案

karthik的答案如果有重复的行将失败在数据框中. 'except'将删除df2中df1中的所有行.

karthik's answer will fail if there are duplicate rows in the dataframe. 'except' will remove all rows in df1 which are in df2 .

val filteredRdd = df.rdd.zipWithIndex().collect { case (r, i) if 10 >= start && i <=20 => r }
val newDf = sqlContext.createDataFrame(filteredRdd, df.schema)

这篇关于如何实现Spark sql分页查询的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！