apache Pig 试图在每个组中获得最大数量 [英] apache Pig trying to get max count in each group

问题描述

我有猪格式的数据

{(group, productId, count)}.

现在我想获得每个组中的最大计数，输出可能如下所示

Now I want to get maximum count in each group and the output might look as follows

{(group, productId, maxCount)}.这是示例输入数据

• (south America,prod1, 45),(south America,prod2, 36), (latin America, prod1, 48),(latin America, prod5,35)

这里是这个输入的输出看起来像

here is the output for this input look like

1. (南美洲, prod1, 45)
2. (北美，prod2, 36)
3. (拉丁美洲, prod1, 48)

有人可以帮助我吗.

推荐答案

根据您的示例输入数据，这应该可以解决问题:

Based on your sample input data, this should do the trick:

data = load 'sf.csv' using PigStorage(',') as (country:chararray, product:chararray, c:int);
g = group data by country;
result = foreach g {
    prods = order data by c desc;
    top_prods = limit prods 1;
    generate flatten(top_prods);
}
dump result;

这按第一列对输入进行分组，然后在嵌套的 foreach 中按计数对每组的产品进行排序，然后取第一个(最高计数).

This groups the input by first column, then in the nested foreach it orders the products per group by count, then takes the first (highest count).

输出:

(latin america,prod1,48)
(south America,prod1,45)

这篇关于apache Pig 试图在每个组中获得最大数量的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！