Java Hadoop Mapper如何发送多个值 [英] How Java Hadoop Mapper can send multiple values

问题描述

我的映射器需要发送以下元组：

 < custID，prodID，rate> 
  

我想发送reducer作为key的custID，并将prodID和rate一起使用，因为它们是缩小阶段所需要的。
这是做这件事的最好方法吗？

  public void map（Object key，Text value，Context context）
 throws IOException，InterruptedException {
 
 String [] col = value.toString（）。split（，）; 
 custID.set（col [0]）; 
 data.set（col [1] +，+ col [2]）; 
 context.write（custID，data）; 
 
 $ b $ public void reduce（Text key，Iterable< Text> values，Context context）
 throws IOException，InterruptedException {（
 
 for（Text val ：values）{
 String [] temp = val.toString（）。split（，）; 
 Text rate = new Text（temp [1]）; 
 result.set（rate）; 
 context.write（key，result）; 
} 
} 
  

解决方案

最简单的我能想到的只是将它们合并成一个单一字符串：

$ $ $ $ $ $ $ output.collect（custID，prodID +，） +率）;

然后，如果备份在减速器上，则拆分。

如果你从你的映射器发布了更多的代码，我们可以举一个更好的例子。

更新：也就是说，你要求最好的方式。最正确的方法可能是创建一个单独的类分组 prodID 和 rate 在一起并发送。

My mapper needs to send the following tuples:

<custID,prodID,rate>

And I want to send to reducer the custID as a key, and as value the prodID and rate together, as they are needed for the reduce phase. Which is the best way of doing this?

public void map(Object key, Text value, Context context) 
        throws IOException, InterruptedException {

    String[] col = value.toString().split(",");
    custID.set(col[0]);
    data.set(col[1] + "," + col[2]);
    context.write(custID, data);
}

public void reduce(Text key, Iterable<Text> values, Context context)
        throws IOException, InterruptedException {

    for (Text val : values) {
        String[] temp = val.toString().split(",");
        Text rate = new Text(temp[1]);
        result.set(rate);
        context.write(key, result);
    }
}

解决方案

The simplest I can think of is just to merge them into a single string:

output.collect(custID, prodID + "," + rate);

Then, split if back up on the reducers.

If you post a little more code from your mapper maybe we could give a better example.

UPDATE: That said, you asked for the best way. The most correct way is probably to create a separate class grouping prodID and rate together and send that.

这篇关于Java Hadoop Mapper如何发送多个值的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！