在猪中正确加载日期时间 [英] properly loading datetime in pig

问题描述

我正在加载一个带有日期时间列和长列的 tsv 文件:

I'm loading a tsv file with a datetime column and long column with:

A = LOAD 'tweets-clean.txt' USING PigStorage('\t') AS (date:datetime, userid:long);
DUMP A;

输入行示例:

Tue Feb 11 05:02:10 +0000 2014  205291417

那一行输出:

, 205291417

我该如何正确执行此操作?

How do I do this properly?

推荐答案

您希望将日期加载为字符数组 (date:chararray)，然后可以使用 FOREACH GENERATE 连同 ToDate Pig 内置功能.

You'd want to load date as a chararray (date:chararray) and then can convert it to to a datetime using FOREACH GENERATE along with the ToDate Pig built-in function.

格式字符串基于 简单日期格式

The format string is based on the SimpleDateFormat

A = LOAD 'tweets-clean.txt' USING PigStorage('\t') AS (date:chararray, userid:long);
B = FOREACH A GENERATE ToDate(date, '<some format string>') AS date, userid;
DUMP B;

这篇关于在猪中正确加载日期时间的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！