在数据框Spark Scala中更改列值 [英] Change column value in a dataframe spark scala

问题描述

这是我目前的数据框外观

This is how my dataframe looks like at the moment

+------------+
|    DATE    |
+------------+
|    19931001|
|    19930404|
|    19930603|
|    19930805|
+------------+

我正在尝试将此字符串值重新格式化为yyyy-mm-dd hh:mm:ss.fff，并将其保留为字符串，而不是日期类型或时间戳.

I am trying to reformat this string value to yyyy-mm-dd hh:mm:ss.fff and keep it as a string not a date type or time stamp.

我该如何使用withColumn方法?

How would I do that using the withColumn method ?

推荐答案

以下是使用UDF和withcolumn的解决方案，我假设您在Dataframe

Here is the solution using UDF and withcolumn, I have assumed that you have a string date field in Dataframe

//Create dfList dataframe
  val dfList = spark.sparkContext
    .parallelize(Seq("19931001","19930404", "19930603", "19930805")).toDF("DATE")


  dfList.withColumn("DATE", dateToTimeStamp($"DATE")).show()

  val dateToTimeStamp = udf((date: String) => {
    val stringDate = date.substring(0,4)+"/"+date.substring(4,6)+"/"+date.substring(6,8)
    val format = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss")
    format.format(new SimpleDateFormat("yyy/MM/dd").parse(stringDate))
  })

这篇关于在数据框Spark Scala中更改列值的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！