Pig - 如何操作和比较日期? [英] Pig - How to manipulate and compare dates?

问题描述

我有一个包含以下条目的文件:

I have a file which contains entries like this:

1,1,07 2012,07 2013,11,blablabla

前两个字段是 id.第三个是开始日期(月年)，第四个是结束日期.第五个字段是这两个日期之间的月数.最后一个字段包含文本.

The two first fields are ids. The third is the begin date(month year) and the fourth is the end date. The fifth field is the number of months btweens these two dates. And the last field contains text.

这是我加载这些数据的猪代码:

Here is my pig code to load this data:

f = LOAD 'file.txt' USING PigStorage(',') AS (id1:int, id2:int, date1:chararray, date2:chararray, duration:int, text:chararray);

我想过滤我的文件，以便只保留 date2 从今天起不到三年的条目.在 Pig 中有可能吗?

I would like to filter my file so that I keep only the entries where date2 is less than three years from today. Is it possible to that in Pig ?

谢谢.

推荐答案

无需编写自定义函数:

在 Pig 0.11 中，您可以使用 ToDate() 函数，然后使用 YearsBetween() 获取 CurrentTime() 和 date2 之间的差异并根据它进行过滤.例如:

In Pig 0.11 you can convert the date2 field from chararray to datetime data type using the ToDate() function, and then get the difference between the CurrentTime() and date2 using YearsBetween() and filter according to it. for example:

g = FILTER f BY YearsBetween(CurrentTime(),ToDate(date2 + ' 01', 'yyyy MM dd'))<3

这篇关于Pig - 如何操作和比较日期?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！