为 spark.createDataFrame 动态设置架构 [英] Dynamically setting schema for spark.createDataFrame

问题描述

所以我试图动态设置架构中的数据类型.

So I am trying to dynamically set the type of data in the schema.

我在stackoverflow上看到了代码schema = StructType([StructField(header[i], StringType(), True) for i in range(len(header))])

I have seen the code schema = StructType([StructField(header[i], StringType(), True) for i in range(len(header))]) on stackoverflow

但是如何将更改添加到条件语句中?

But how can I add change this into a conditional statement?

如果header在list1中，则IntergerType，如果在list2中则为DoubleType，否则为StringType?

If header is in list1 then IntergerType, if in list2 then DoubleType, else StringType for example?

推荐答案

一位同事为我解答了这个问题

A colleague answered this for me

schema = StructType([
    StructField(header[i], DateType(), True) 
      if header[i] in dateFields 
      else StructField(header[i], StringType(), True)
      for i in range(len(header))])

这篇关于为 spark.createDataFrame 动态设置架构的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！