找出分区编号/编号 [英] Find out the partition no/id

问题描述

Spark中是否有一种方法(方法)来查找分区ID/否

Is there a way(A method) in Spark to find out the Parition ID/No

在此举个例子

val input1 = sc.parallelize(List(8, 9, 10), 3)

val res = input1.reduce{ (x, y) => println("Inside partiton " + ???)

                               x + y)}

我想在 ??? 中添加一些代码以打印分区ID/否

I would like to put some code in ??? to print the Partition ID / No

推荐答案

实际上， mapParitionsWithIndex 将为您提供迭代器&分区索引.(当然这与reduce并不相同，但是您可以将其结果与 aggregate 结合起来.)

Indeed, the mapParitionsWithIndex will give you an iterator & the partition index. (This isn't the same as reduce of course, but you could combine the result of that with aggregate).

这篇关于找出分区编号/编号的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！