到目前为止的Spark DataFrame列字符串 [英] spark dataframe column string to date

问题描述

我想将"yyyyMMdd"的spark数据框字符串列转换为带有spark会话(火花)的日期格式-而不是spark上下文.

I would like to convert a spark dataframe string column of 'yyyyMMdd' to date format with spark session (spark) - not spark context.

由于我不使用Spark上下文(sc)，因此无法使用以下代码，尽管它可以精确地完成我想做的事情:

Since I'm not working with spark context (sc), I cannot use the following code, although it would precisly do what I'd like it to do:

.withColumn("column1",DF.to_date(F.col("column1"),"yyyyMMdd"))

由于我不想将列转换为时间戳，所以我也不想使用以下代码:

As I do not want to convert the column to timestamp, I also don't want to use the following code:

.withColumn("column1", unix_timestamp(col("column1"), "yyyyMMdd").cast("timestamp"))

最终目标是将前一个字符串列替换为日期格式的列.

The final goal is to replace the former string column by the date formatted column.

非常感谢！

推荐答案

以下代码可以正常工作:

the following code works fine:

.withColumn("column1", to_date(DF["column1"], 'yyyyMMdd'))

感谢您的关注！

这篇关于到目前为止的Spark DataFrame列字符串的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！