takeOrdered降序Pyspark [英] takeOrdered descending Pyspark

问题描述

我想按值对K/V对进行排序，然后取最大的五个值.我设法通过在第一个地图上还原K/V，然后在FALSE中按降序排序，然后将key.value反转为原始(第二个地图)，然后取前5个为大块头，代码是这样的:

RDD.map(lambda x:(x[1],x[0])).sortByKey(False).map(lambda x:(x[1],x[0])).take(5)

我知道pySpark上有一个takeOrdered操作，但是我只设法对值进行排序(而不对键进行排序)，我不知道如何进行降序排序:

RDD.takeOrdered(5,key = lambda x: x[1])

解决方案

按键排序(升序):

RDD.takeOrdered(5, key = lambda x: x[0])

按键排序(降序):

RDD.takeOrdered(5, key = lambda x: -x[0])

按值排序(升序):

RDD.takeOrdered(5, key = lambda x: x[1])

按值排序(降序):

RDD.takeOrdered(5, key = lambda x: -x[1])

i would like to sort K/V pairs by values and then take the biggest five values. I managed to do this with reverting K/V with first map, sort in descending order with FALSE, and then reverse key.value to the original (second map) and then take the first 5 that are the bigget, the code is this:

RDD.map(lambda x:(x[1],x[0])).sortByKey(False).map(lambda x:(x[1],x[0])).take(5)

i know there is a takeOrdered action on pySpark, but i only managed to sort on values (and not on key), i don't know how to get a descending sorting:

RDD.takeOrdered(5,key = lambda x: x[1])

解决方案

Sort by keys (ascending):

RDD.takeOrdered(5, key = lambda x: x[0])

Sort by keys (descending):

RDD.takeOrdered(5, key = lambda x: -x[0])

Sort by values (ascending):

RDD.takeOrdered(5, key = lambda x: x[1])

Sort by values (descending):

RDD.takeOrdered(5, key = lambda x: -x[1])

这篇关于takeOrdered降序Pyspark的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！