Spark-Datediff几个月了? [英] Spark - Datediff for months?

问题描述

有没有一种好方法可以在几个月内使用datediff?需要说明的是:datediff方法采用两列，并返回在两个日期之间经过的天的数量.我想在个月内获得.我想在我的函数中有一个参数，该参数可以告诉我检查过去20个月，36个月(无论几个月)中的数据.如果我只是做datediff并将结果除以30(或31)，则结果不是很准确.我可以使用30.4166667(= 365天/12个月)，但这在较短的时间内也不是很准确.那么，关于如何使用datediff来获取收益的任何技巧? SQL就像SELECT DATEDIFF(month, '2005-12-31 23:59:59.9999999', '2006-01-01 00:00:00.0000000');一样，我正在Spark中寻找类似的东西.

Is there a good way to use datediff with months? To clarify: the datediff method takes two columns and returns the number of days that have passed between the two dates. I'd like to have that in months. I want to have a parameter in my function that can I tell to check data say from the last 20, 36, whatever months. If I just do datediff and divide the result with 30 (or 31), than the result is not quite accurate. I could use 30.4166667 (= 365 days/12 months), but that is not quite accurate either for shorter periods. So, any tips on how to use datediff to be able to get months out of it? SQL has it like SELECT DATEDIFF(month, '2005-12-31 23:59:59.9999999', '2006-01-01 00:00:00.0000000');, I'm looking for something like this in Spark.

推荐答案

您可以尝试

You can try months_between:

import org.apache.spark.sql.functions.*
DataFrame newDF = df.withColumn("monthDiff", months_between(col("col1"), col("col2"))

这篇关于Spark-Datediff几个月了?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！