猪:旋转&和 3 关系 [英] Pig: Pivoting & Sum 3 relations

问题描述

我有 3 种不同的关系，如下所述 &我可以使用 UDF 获得输出，但在 PIG 中寻找实现.在论坛中提到了其他东西，但没有对这个问题有具体的想法.

i have 3 different relations as mentioned below & i can get the output using UDFs but looking for implementation in PIG. Referred other stuff in the forums but not getting concrete idea over this problem.

过程:

FN1,10
FN2,20
FN3,23
FN4,25
FN5,15
FN7,40
FN10,56

雷杰:

FN1,12
FN2,13
FN3,33
FN6,60
FN8,23
FN9,44
FN10,4

AllFN:

FN1
FN2
FN3
FN4
FN5
FN6
FN7
FN8
FN9
FN10

所需的输出是:

FN1,10,12,22
FN2,20,13,33
FN3,23,33,56
FN4,25,0,25
FN5,15,0,15
FN6,0,60,60
FN7,40,0,40
FN8,0,23,23
FN9,0,44,44
FN10,56,4,60

推荐答案

假设你的关系在 test.txt test2.txt 和 test3.txt

Asumming your relations are in test.txt test2.txt and test3.txt

A = LOAD 'test.txt' using PigStorage(',');
B = LOAD 'test2.txt' using PigStorage(',');
C = LOAD 'test3.txt' using PigStorage(',');
D = COGROUP A by $0, B by $0;
E = COGROUP C by $0, D by $0;
F = FOREACH E generate $0, FLATTEN(D.A), FLATTEN(D.B);
G = FOREACH F generate $0, $1.$1, $2.$1;
H = FOREACH G generate $0, FLATTEN((IsEmpty($1)?null:$1)), FLATTEN((IsEmpty($2)?null:$2));
I = foreach H generate $0, ($1 is null?0:$1),($2 is null?0:$2),($1 is null?0:$1)+($2 is null?$0:$2);
dump I;

输出

(FN1,10,12,22)
(FN2,20,13,33)
(FN3,23,33,56)
(FN4,25,0,)
(FN5,15,0,)
(FN6,0,60,60)
(FN7,40,0,)
(FN8,0,23,23)
(FN9,0,44,44)
(FN10,56,4,60)

这篇关于猪:旋转&和 3 关系的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！