在pyspark中按行连接字符串 [英] Concatenating string by rows in pyspark

问题描述

我有一个 pyspark 数据框

I am having a pyspark dataframe as

DOCTOR | PATIENT
JOHN   | SAM
JOHN   | PETER
JOHN   | ROBIN
BEN    | ROSE
BEN    | GRAY

并且需要按行连接患者姓名，以便得到如下输出:

and need to concatenate patient names by rows so that I get the output like:

DOCTOR | PATIENT
JOHN   | SAM, PETER, ROBIN
BEN    | ROSE, GRAY

有人可以帮助我在 pyspark 中创建此数据框吗?

Can anybody help me regarding creating this dataframe in pyspark ?

提前致谢.

推荐答案

我能想到的最简单的方法就是使用collect_list

The simplest way I can think of is to use collect_list

import pyspark.sql.functions as f
df.groupby("col1").agg(f.concat_ws(", ", f.collect_list(df.col2)))

这篇关于在pyspark中按行连接字符串的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！