将宽数据帧转置为长数据帧 [英] Transpose wide dataframe to long dataframe

问题描述

我有一个数据框看起来像:

I have a data frame looks like:

Region, 2000Q1, 2000Q2, 2000Q3, ...
A, 1,2,3,...

我想通过区域"将这张宽表转换为长表.所以最终产品将如下所示:

I want to transpose this wide table to a long table by 'Region'. So the final product will look like:

Region, Time, Value
A, 2000Q1,1
A, 2000Q2, 2
A, 2000Q3, 3
A, 2000Q4, 4
....

原始表的列数组非常广泛，但聚合级别始终为区域，其余列设置为转置.

The original table has a very wide array of columns but the aggregation level is always region and remaining columns are set to be tranposed.

你知道一个简单的方法或函数来做到这一点吗?

Do you know an easy way or function to do this?

推荐答案

类似但改进了列计算.

cols = df.columns
cols.remove('Region')

import pyspark.sql.functions as f

df.withColumn('array', f.explode(f.arrays_zip(f.array(*map(lambda x: f.lit(x), cols)), f.array(*cols), ))) \
  .select('Region', 'array.*') \
  .toDF('Region', 'Time', 'Value') \
  .show(30, False)

+------+------+-----+
|Region|Time  |Value|
+------+------+-----+
|A     |2000Q1|1    |
|A     |2000Q2|2    |
|A     |2000Q3|3    |
|A     |2000Q4|4    |
|A     |2000Q5|5    |
+------+------+-----+

附言不要接受这个作为答案:)

p.s. Don't accept this as an answer :)

这篇关于将宽数据帧转置为长数据帧的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！