Spark SQL 来分解结构数组 [英] Spark SQL to explode array of structure

问题描述

我有以下 JSON 结构，我试图将其转换为每个元素作为列的结构，如下所示使用 Spark SQL.爆炸(控制)不起作用.有人可以建议一种方法吗?

I have the below JSON structure which I am trying to convert to a structure with each element as column as shown below using Spark SQL. Explode(control) is not working. Can someone please suggest a way to do this?

输入:

{
"control" : [
    {
      "controlIndex": 0,
      "customerValue": 100.0,
      "guid": "abcd",
      "defaultValue": 50.0,
      "type": "discrete"
    },
    {
      "controlIndex": 1,
      "customerValue": 50.0,
      "guid": "pqrs",
      "defaultValue": 50.0,
      "type": "discrete"
    }
  ]
}

所需的输出:

controlIndex customerValue guid defaultValult  type

0            100.0         abcd   50.0         discrete

1            50.0          pqrs   50.0         discrete

推荐答案

除了 Paul Leclercq 的回答之外，这里还有一些可行的方法.

Addition to Paul Leclercq's answer, here is what can work.

import org.apache.spark.sql.functions.explode   
df.select(explode($"control").as("control")).select("control.*")

这篇关于Spark SQL 来分解结构数组的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！