如何将嵌套列添加到 DataFrame [英] How to add a nested column to a DataFrame

问题描述

我有一个数据框 df 具有以下架构:

I have a dataframe df with the following schema:

root
 |-- city_name: string (nullable = true)
 |-- person: struct (nullable = true)
 |    |-- age: long (nullable = true)
 |    |-- name: string (nullable = true)

我想要做的是在我的 person 结构中添加一个嵌套列，比如 car_brand.我该怎么做?

What I want to do is add a nested column, say car_brand to my person structure. How would I do it?

预期的最终架构如下所示:

The expected final schema would look like this:

root
 |-- city_name: string (nullable = true)
 |-- person: struct (nullable = true)
 |    |-- age: long (nullable = true)
 |    |-- name: string (nullable = true)
 |    |-- car_brand: string (nullable = true)

推荐答案

您可以解压结构体并将其添加到新结构体中，同时包括新列.例如，将bmw"添加到数据框中的所有人中，如下所示:

You can unpack the struct and add it to a new one, including the new column at the same time. For example, adding "bmw" to all persons in the dataframe be done like this:

df.withColumn("person", struct($"person.*", lit("bmw").as("car_brand")))

这篇关于如何将嵌套列添加到 DataFrame的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！