标量只能与 PIG 中的投影一起使用 [英] scalars can only be used with projection in PIG

问题描述

标量只能与投影一起使用 我在使用 foreach 时遇到此错误.我该如何解决此错误?我如何在 foreach 中使用 LIMIT ?请推荐一些提前致谢..

scalars can only be used with projection i am getting this error while using foreach.How can i resolved this error ? how can i use LIMIT within foreach ? please suggest some thanks in advance..

编辑(Tichdroma):从评论中复制代码

A = LOAD 'part-r-00000';
G = Group A by ($0,$2 );
Y = foreach G generate FLATTEN(group), FLATTEN($1);
sorted = order Y by $0 ASC, $1 DESC;
X = foreach Y {
  lim = LIMIT sorted 3;
  generate lim;
};
Dump x;

推荐答案

LIMIT 在 Pig 0.9 的 FOREACH nested_op.

LIMIT is available in Pig 0.9 in the FOREACH nested_op.

如果你想要每个组的前 N ​​个元素，你可能想尝试对每个元素进行迭代并单独排序和限制它们:

If you want the top N element of each group, you might want to try to iterate on each one and individually sort and limit them:

A = LOAD 'part-r-00000';
G = GROUP A by ($0, $2);
X = FOREACH G {
  sorted = ORDER A by $0 ASC, $1 DESC;
  lim = LIMIT sorted 3;
  GENERATE lim;
};
DUMP X;

请注意，TOP 在您只拥有一列可比较的值(不是在这种情况下).

Notice that TOP can be efficient when you just have a column of comparable values (not in this case).

这篇关于标量只能与 PIG 中的投影一起使用的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！