根据另一列将值映射到特定列 [英] Mapping a value into a specific column based on annother column

问题描述

我有以下问题:

• A DataFrame 包含带有字符串 A、B 或 C 的 col1.
• 第二个 col2 带有 Integer.
• 以及另外三个列col3、col4 和col5(这些列也被命名为A、B 和 C).

• A DataFrame containing col1 with strings A, B, or C.
• A second col2 with an Integer.
• And three other columns col3, col4 and col5 (these columns are also named A, B, and C).

因此，

 col1 - col2 - A (col3) - B (col4) - C (col5)
|--------------------------------------------
   A      6
   B      5
   C      6

应该得到

 col1 - col2 - A (col3) - B (col4) - C (col5)
|--------------------------------------------
   A      6       6
   B      5                  5
   C      6                              6

现在我想遍历每一行并根据 col1 中的条目将 col2 中的整数分配给 A、B 或 C 列.

Now I would like to go through each row and assign the integer in col2 to the column A, B or C based on the entry in col1.

我如何实现这一目标?

df.withColumn() 我不能使用(或者至少我不知道为什么)，对于 val df2 = df.map(x => x ).

df.withColumn() I cannot use (or at least I do not know why) and the same holds for val df2 = df.map(x => x ).

期待您的帮助并提前致谢！

Looking forward to you help and thanks in advance!

最好的，肯

推荐答案

创建键和目标列之间的映射:

Create a mapping between key and target column:

val mapping = Seq(("A", "col3"), ("B", "col4"), ("C", "col5"))

用它来生成列序列:

import org.apache.spark.sql.functions.when

val exprs = mapping.map { case (key, target) => 
  when($"col1" === key, $"col2").alias(target) }

前置星号并选择:

val df = Seq(("A", 6), ("B", 5), ("C", 6)).toDF("col1", "col2")
df.select($"*" +: exprs: _*)

结果是:

+----+----+----+----+----+
|col1|col2|col3|col4|col5|
+----+----+----+----+----+
|   A|   6|   6|null|null|
|   B|   5|null|   5|null|
|   C|   6|null|null|   6|
+----+----+----+----+----+

这篇关于根据另一列将值映射到特定列的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！