在火花对RDD中按值排序 [英] Order by value in spark pair RDD

问题描述

我有一个火花对RDD(键，计数)，如下所示

I have a spark pair RDD (key, count) as below

Array[(String, Int)] = Array((a,1), (b,2), (c,1), (d,3))

使用spark scala API如何获取按值排序的新对RDD?

Using spark scala API how to get a new pair RDD which is sorted by value?

必填结果:Array((d,3), (b,2), (a,1), (c,1))

推荐答案

这应该有效:

//Assuming the pair's second type has an Ordering, which is the case for Int
rdd.sortBy(_._2) // same as rdd.sortBy(pair => pair._2)

(尽管有联系，您可能也想把钥匙考虑在内.)

(Though you might want to take the key to account too when there are ties.)

这篇关于在火花对RDD中按值排序的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！