如何在猪/蜂房的相应的几栏中进行移调 [英] how to do Transpose in corresponding few columns in pig/hive
问题描述
处理数据我得到以下要求
id jan feb march
1 j1 f1 m1
2 j2 f2 m2
3 j3 f3 m3
其中我需要将它转换为第一列,所以它看起来像 -
id值月份
1 j1 jan
1 f1 feb
1 m1 march
2 j2 jan
2 f2 feb
2 m2 march
3 j3 jan
3 f3 feb
3 m3 march
我曾尝试过使用java,但为了进入分布式模式任何方式在猪/蜂房做到这一点。
预先感谢您的帮助!!
Pig没有任何内置函数解决你的需求,但你可以尝试下面的方法,我想它会适用于你。
input.txt
1 j1 f1 m1
p
2 j2 f2 m2
3 j3 f3 m3
$ bPigScript:
A = LOAD'input.txt'使用PigStorage()AS(id,month1,month2,month3);
B = FOREACH A GENERATE FLATTEN(TOBAG(TOTUPLE(id,month1,'jan'),TOTUPLE(id,month2,'feb'),TOTUPLE(id,month3,'mar')));
DUMP B;
输出:
(1,j1,jan)
(1,f1,feb)
(1,m1,mar)
(2,j2, jan)
(2,f2,feb)
(2,m2,mar)
(3,j3,jan)
(3,f3,feb)
(3,m3,mar)
I was wondering is it possible to do transposition corresponding few columns in pig/hive.
as dealing with data i got below requirement
id jan feb march
1 j1 f1 m1
2 j2 f2 m2
3 j3 f3 m3
where i need to transpose it against first column, so it would look like -
id value month
1 j1 jan
1 f1 feb
1 m1 march
2 j2 jan
2 f2 feb
2 m2 march
3 j3 jan
3 f3 feb
3 m3 march
I have tried this with java, but to get it into distributed mode is there any way to do it in pig/hive.
appreciating your help in advance!!
Pig doesn't have any built-in function to solve your requirement, but you can try the below approach, i guess it will work for you.
input.txt
1 j1 f1 m1
2 j2 f2 m2
3 j3 f3 m3
PigScript:
A = LOAD 'input.txt' USING PigStorage() AS (id,month1,month2,month3);
B = FOREACH A GENERATE FLATTEN(TOBAG(TOTUPLE(id,month1,'jan'),TOTUPLE(id,month2,'feb'),TOTUPLE(id,month3,'mar')));
DUMP B;
Output:
(1,j1,jan)
(1,f1,feb)
(1,m1,mar)
(2,j2,jan)
(2,f2,feb)
(2,m2,mar)
(3,j3,jan)
(3,f3,feb)
(3,m3,mar)
这篇关于如何在猪/蜂房的相应的几栏中进行移调的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!