Apache Pig Rank函数的用法 [英] Usage of Apache Pig rank function
问题描述
Am使用Pig 0.11.0等级函数,并为数据中的每个id生成等级. 我需要以特定方式对数据进行排名.我希望重置等级并为每个新ID从1开始.
Am using Pig 0.11.0 rank function and generating ranks for every id in my data. I need ranking of my data in a particular way. I want the rank to reset and start from 1 for every new ID.
是否可以直接将rank函数用于同一个? 任何提示将不胜感激.
Is it possible to use the rank function directly for the same? Any tips would be appreciated.
数据:
id,rating
X001, 9
X001, 9
X001, 8
X002, 9
X002, 7
X002, 6
X002, 5
X003, 8
X004, 8
X004, 7
X004, 7
X004, 4
在使用排名函数时,例如: op =按ID,得分对数据进行排名;
On using rank function like: op = rank data by id,score;
我得到这个输出
rank,id,rating
1, X001, 9
1, X001, 9
2, X001, 8
3, X002, 9
4, X002, 7
5, X002, 6
6, X002, 5
7, X003, 8
8, X004, 8
9, X004, 7
9, X004, 7
10, X004, 4
所需的O/P:
rank,id,rating
1, X001, 9
1, X001, 9
2, X001, 8
1, X002, 9
2, X002, 7
3, X002, 6
4, X002, 5
1, X003, 8
1, X004, 8
2, X004, 7
2, X004, 7
3, X004, 4
推荐答案
You can group your data by id then use the UDF Enumerate (DataFu) to append an index to each tuple of the bags.
register datafu-1.1.0.jar;
define Enumerate datafu.pig.bags.Enumerate('1');
data = load 'data' using PigStorage(',') as (id:chararray, rating:int);
data = group data by id;
data = foreach data {
sorted = order data by rating DESC;
generate group, sorted;
}
data = foreach data generate FLATTEN(Enumerate(sorted));
data = foreach data generate $2, $0, $1;
dump data;
可以从Maven Central存储库下载
DataFu jar文件: http ://search.maven.org/#search | ga | 1 | g%3A%22com.linkedin.datafu%22
DataFu jar file can be downloaded from Maven Central repository: http://search.maven.org/#search|ga|1|g%3A%22com.linkedin.datafu%22
这篇关于Apache Pig Rank函数的用法的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!