PIG中整套记录的最大/最小值 [英] Max/Min for whole sets of records in PIG

问题描述

我有一组记录，我从一个文件加载，我需要做的第一件事是获取列的最大值和最小值。
在SQL中，我会用这样的子查询来做到这一点：

I have a set set of records that I am loading from a file and the first thing I need to do is get the max and min of a column. In SQL I would do this with a subquery like this:

   select c.state, c.population, 
(select max(c.population) from state_info c) as max_pop, 
(select min(c.population) from state_info c) as min_pop
from state_info c

我假设在PIG中必须有一个简单的方法来做到这一点，但我很难找到它。它有一个MAX和MIN函数，但是当我尝试执行以下操作时，它不起作用：

I assume there must be an easy way to do this in PIG as well but I'm having trouble finding it. It has a MAX and MIN function but when I tried doing the following it didn't work:

records=LOAD '/Users/Winter/School/st_incm.txt'  AS (state:chararray, population:int);
with_max = FOREACH records GENERATE state, population, MAX(population);

这不起作用。我有更好的运气为每行添加一个具有相同值的额外列，然后将它们分组到这一列。然后获得新组的最大值。这看起来像是一种令人费解的方式来获得我想要的东西，所以我想我会问，如果有人知道更简单的方法。

This didn't work. I had better luck adding an extra column with the same value to each row and then grouping them on that column. Then getting the max on that new group. This seems like a convoluted way of getting what I want so I thought I'd ask if anyone knows a simpler way.

预先感谢您的帮助。

推荐答案

正如您所说，您需要将所有数据组合在一起，但如果使用全部汇总。

As you said you need to group all the data together but no extra column is required if you use GROUP ALL.

猪

records = LOAD 'states.txt'  AS (state:chararray, population:int);
records_group = GROUP records ALL;
with_max = FOREACH records_group 
           GENERATE
               FLATTEN(records.(state, population)), MAX(records.population);

输入

Input

CA  10
VA  5
WI  2

p>

Output

(CA,10,10)
(VA,5,10)
(WI,2,10)

这篇关于PIG中整套记录的最大/最小值的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！