Apache Spark Dataframe Groupby agg()用于多列 [英] Apache Spark Dataframe Groupby agg() for multiple columns

问题描述

我有一个3列的DataFrame，即Id, First Name, Last Name

我想在Id的基础上应用GroupBy，并希望收集First Name, Last Name列作为列表.

I want to apply GroupBy on the basis of Id and want to collect First Name, Last Name column as list.

示例:-我有一个这样的DF

Example :- I have a DF like this

+---+-------+--------+
|id |fName  |lName   |
+---+-------+--------+
|1  |Akash  |Sethi   |
|2  |Kunal  |Kapoor  |
|3  |Rishabh|Verma   |
|2  |Sonu   |Mehrotra|
+---+-------+--------+

我想要这样的输出

+---+-------+--------+--------------------+
|id |fname           |lName               |
+---+-------+--------+--------------------+
|1  |[Akash]         |[Sethi]             |
|2  |[Kunal, Sonu]   |[Kapoor, Mehrotra]  |
|3  |[Rishabh]       |[Verma]             |
+---+-------+--------+--------------------+

预先感谢

推荐答案

您可以聚合多个列，如下所示:

You can aggregate multiple columns like this:

df.groupBy("id").agg(collect_list("fName"), collect_list("lName"))

它将给您预期的结果.

这篇关于Apache Spark Dataframe Groupby agg()用于多列的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！