猪:在嵌套的foreach中获取索引 [英] Pig: Get index in nested foreach

问题描述

我有一个猪脚本，其代码如下:

I have a pig script with code like :

scores = LOAD 'file' as (id:chararray, scoreid:chararray, score:int);
scoresGrouped = GROUP scores by id;
top10s = foreach scoresGrouped{
    sorted = order scores by score DESC;
    sorted10 = LIMIT sorted 10;
    GENERATE group as id, sorted10.scoreid as top10candidates;
};

给我一​​个像这样的包

It gets me a bag like

 id1, {(scoreidA),(scoreidB),(scoreIdC)..(scoreIdFoo)}

但是，我也希望包括项目索引，所以我得到的结果是

However, I wish to include the index of items as well, so I'd have results like

 id1, {(scoreidA,1),(scoreidB,2),(scoreIdC,3)..(scoreIdFoo,10)}

是否可以将索引以某种方式包含在嵌套的foreach中，还是我必须编写自己的UDF以在之后添加它?

Is it possible to include the index somehow in the nested foreach, or would I have to write my own UDF to add it in afterwards?

推荐答案

要为包中的元素建立索引，可以使用 DataFu 项目:

For indexing elements in a bag you may use the Enumerate UDF from LinkedIn's DataFu project:

register '/path_to_jar/datafu-0.0.4.jar';
define Enumerate datafu.pig.bags.Enumerate('1');
scores = ...
...
result = foreach top10s generate id, Enumerate(top10candidates);

这篇关于猪:在嵌套的foreach中获取索引的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！