Pig:在嵌套的foreach中获取索引 [英] Pig: Get index in nested foreach

问题描述

我有一个猪脚本，代码如下:

I have a pig script with code like :

scores = LOAD 'file' as (id:chararray, scoreid:chararray, score:int);
scoresGrouped = GROUP scores by id;
top10s = foreach scoresGrouped{
    sorted = order scores by score DESC;
    sorted10 = LIMIT sorted 10;
    GENERATE group as id, sorted10.scoreid as top10candidates;
};

它给了我一个包

 id1, {(scoreidA),(scoreidB),(scoreIdC)..(scoreIdFoo)}

但是，我也希望包含项目的索引，所以我会得到类似的结果

However, I wish to include the index of items as well, so I'd have results like

 id1, {(scoreidA,1),(scoreidB,2),(scoreIdC,3)..(scoreIdFoo,10)}

是否可以在嵌套的 foreach 中以某种方式包含索引，或者我是否必须编写自己的 UDF 以在之后添加它?

Is it possible to include the index somehow in the nested foreach, or would I have to write my own UDF to add it in afterwards?

推荐答案

要索引包中的元素，您可以使用 枚举 LinkedIn DataFu UDFa> 项目:

For indexing elements in a bag you may use the Enumerate UDF from LinkedIn's DataFu project:

register '/path_to_jar/datafu-0.0.4.jar';
define Enumerate datafu.pig.bags.Enumerate('1');
scores = ...
...
result = foreach top10s generate id, Enumerate(top10candidates);

这篇关于Pig:在嵌套的foreach中获取索引的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！