使用PIG加载日期时间格式文件 [英] Loading datetime format files using PIG
问题描述
ravi，savings，avinash，2，char，33，F， 22,44,12,13,33,44,22,11,10,22,2006-01-23
avinash，current，sandeep，3，char，44，M，33,11,10,12 ，33,22,39,12,23,19,2001-02-12
supreeth，储蓄，prabhash，4，char，55，F，22,12,23,12,44,56,7， 88,34,23,1995-03-11
lavi，current，nirmesh，5，char，33，M，11,10,33,34,56,78,54,23,445,66,1999-06 -15
Venkat，储蓄，兔子，6，char，11，F，99,12,34,55,33,23,45,66,23,23,2016-05-18
最后一列（例如：2006-01-23）是日期。
我正尝试使用PIG以下命令加载上述数据。
以下是我用来加载文件的代码。
file = LOAD'FI_USER_CREDS_TBL_T.txt'
使用PigStorage（'，'）AS（USER_ID：chararray，
ROLE_ID：chararray，
USER_PW：chararray，
NUM_PWD_HISTORY：int，
PWD_HISTORY：chararray，
PWD_LAST_MOD_TIME：int，
NUM_PWD_ATTEMPTS：int，
NEW_USER_FLG：chararray，
LOGIN_TIME_LOW：int，
LOGIN_TIME_HIGH：int，
DISABLED_FROM_DATE：int，
DISABLED_UPTO_DATE：int，
PW_EXPY_DATE：int，
ACCT_EXPY_DATE：int ，
ACCT_INACTIVE_DAYS：int，
LAST_ACCESS_TIME：int，
TS_CNT：int，
DTL__CAPXTIMESTAMP：int，
ETL_INSERT_DATE：datetime）;
但是它不会读取日期列，而是在使用dump file命令后输出以下输出。 / p>
（ravi，savings，avinash，2，char，33，，22,44,12,13,33,44,22 ，11,10,22 ,,）
（avinash，current，sandeep，3，char，44，，33,11,10,12,33,22,39,12,23,19 ,, $） b $ b（supreeth，savings，prabhash，4，char，55，，22,12,23,12,44,56,7,88,34,23 ,,）
（lavi，current，nirmesh， 5，char，33，，11,10,33,34,56,78,54,23,445,66 ,,）
（Venkat，储蓄，兔子，6，char，11，，99,12,34 ，55,33,23,45,66,23,23 ,,）
我怎么读
谢谢。
加载日期为chararray，然后转换为日期格式
file2 = FOREACH文件GENERATE ToDate（date，'dd / MM / yyyy'）AS date，....
试试这个链接以供参考，
http://pig.apache.org/docs/r0.11.0/api/org/apache/pig/builtin/ToDate.html
或
http：// docs .oracle.com / javase / 7 / docs / api / java / text / SimpleDateFormat.html
I have a dataset in the following way.
ravi,savings,avinash,2,char,33,F,22,44,12,13,33,44,22,11,10,22,2006-01-23
avinash,current,sandeep,3,char,44,M,33,11,10,12,33,22,39,12,23,19,2001-02-12
supreeth,savings,prabhash,4,char,55,F,22,12,23,12,44,56,7,88,34,23,1995-03-11
lavi,current,nirmesh,5,char,33,M,11,10,33,34,56,78,54,23,445,66,1999-06-15
Venkat,savings,bunny,6,char,11,F,99,12,34,55,33,23,45,66,23,23,2016-05-18
the last column(example:2006-01-23) is date. I am trying to load the above data with following command using PIG. Following is the code which i am using to load the file.
file = LOAD 'FI_USER_CREDS_TBL_T.txt'
USING PigStorage(',') AS (USER_ID:chararray,
ROLE_ID:chararray,
USER_PW:chararray,
NUM_PWD_HISTORY:int,
PWD_HISTORY:chararray,
PWD_LAST_MOD_TIME:int,
NUM_PWD_ATTEMPTS:int,
NEW_USER_FLG:chararray,
LOGIN_TIME_LOW:int,
LOGIN_TIME_HIGH:int,
DISABLED_FROM_DATE:int,
DISABLED_UPTO_DATE:int,
PW_EXPY_DATE:int,
ACCT_EXPY_DATE:int,
ACCT_INACTIVE_DAYS:int,
LAST_ACCESS_TIME:int,
TS_CNT:int,
DTL__CAPXTIMESTAMP:int,
ETL_INSERT_DATE:datetime);
But it is not reading the date column instead it is giving following output after using dump file command.
(ravi,savings,avinash,2,char,33,,22,44,12,13,33,44,22,11,10,22,,)
(avinash,current,sandeep,3,char,44,,33,11,10,12,33,22,39,12,23,19,,)
(supreeth,savings,prabhash,4,char,55,,22,12,23,12,44,56,7,88,34,23,,)
(lavi,current,nirmesh,5,char,33,,11,10,33,34,56,78,54,23,445,66,,)
(Venkat,savings,bunny,6,char,11,,99,12,34,55,33,23,45,66,23,23,,)
how can i read the date column.
Kindly help me in this regard.
Thank-you.
Load date as chararray and then convert into date format
like:
file2 = FOREACH file GENERATE ToDate(date, 'dd/MM/yyyy') AS date,....
Try this link for reference, http://pig.apache.org/docs/r0.11.0/api/org/apache/pig/builtin/ToDate.html OR http://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html
这篇关于使用PIG加载日期时间格式文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
