读取包装在Hadoop ArrayWritable中的值 [英] Read values wrapped in Hadoop ArrayWritable

问题描述

我是Hadoop和Java的新手。我的映射器输出文本和Arraywritable。我无法读取ArrayWritable值。 Unbale将.get（）值转换为整数。映射器和减速器代码已附加。有人可以帮我纠正我的reducer代码，以读取ArrayWritable值吗？

  public static class Temp2Mapper extends Mapper< LongWritable， Text，Text，ArrayWritable> {
 private static final int MISSING = 9999; 
 
 @Override public void map（LongWritable key，Text value，Context context）throws IOException，InterruptedException {
 String line = value.toString（）; 
 String date = line.substring（07,14）; 
 int maxTemp，minTemp，avgTemp; 
 
 IntArrayWritable carrier = new IntArrayWritable（）; 
 IntWritable innercarrier [] = new IntWritable [3]; 
 maxTemp = Integer.parseInt（line.substring（39,45））; 
 minTemp = Integer.parseInt（line.substring（47,53））; 
 avgTemp = Integer.parseInt（line.substring（63,69））; 
 if（maxTemp！= MISSING）
 innercarrier [0] = new IntWritable（maxTemp）; //最高温度
 if（minTemp！= MISSING）
 innercarrier [1] = new IntWritable（minTemp）; //最低温度
 if（avgTemp！= MISSING）
 innercarrier [2] = new IntWritable（avgTemp）; // 24小时平均温度
 
 carrier.set（innercarrier）; 
 context.write（new Text（date），carrier）; //输出文本和ArrayWritable 
 
 
 
 public static class Temp2Reducer 
 extends Reducer< Text，ArrayWritable，Text，IntWritable> {
 @Override public void reduce（Text key，Iterable< ArrayWritable> values，Context context）
 throws IOException，InterruptedException {
 
 int max = Integer.MIN_VALUE; 
 int [] arr = new int [3]; （ArrayWritable val：values）
 
 {
 arr =（Int）val.get（）; //错误：不能转换可写入int 
 max = Math.max（max，arr [0]）; 
} 
 
 context.write（key，new IntWritable（max））; 
} 
 
} 
  

解决方案

< p> get 方法返回一个 Writable 的数组。

你不能投射一个 Writable 到 int 。你可以做的是：

1. 迭代这个数组
   
2. Writable ）到 IntWritable


3. 使用 IntWritable＃get 方法来获得 int 值。 / li>
   

  for（ArrayWritable val：values）{
 for（可写可写：val.get（））{//迭代
 IntWritable intWritable =（IntWritable）可写; // cast 
 int value = intWritable.get（）; //获得
 //用int值完成你的工作
} 
} 
  

I am new to Hadoop and Java. My mapper outputs text and Arraywritable. I having trouble to read ArrayWritable values. Unbale to cast .get() values to integer. Mapper and reducer code are attached. Can someone please help me to correct my reducer code in order to read ArrayWritable values?

public static class Temp2Mapper extends Mapper<LongWritable, Text, Text, ArrayWritable>{
    private static final int MISSING=9999;

    @Override public void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException{
        String line = value.toString();
        String date = line.substring(07,14);
        int maxTemp,minTemp,avgTemp;

        IntArrayWritable carrier = new IntArrayWritable();
        IntWritable innercarrier[] = new IntWritable[3];
        maxTemp=Integer.parseInt(line.substring(39,45));
        minTemp=Integer.parseInt(line.substring(47,53));
        avgTemp=Integer.parseInt(line.substring(63,69));
        if (maxTemp!= MISSING)
        innercarrier[0]=new IntWritable(maxTemp); // maximum Temperature
        if (minTemp!= MISSING)
        innercarrier[1]=new IntWritable(minTemp); //minimum temperature
        if (avgTemp!= MISSING)
        innercarrier[2]=new IntWritable(avgTemp); // average temperature of 24 hours

        carrier.set(innercarrier);
        context.write(new Text(date), carrier); // Output Text and ArrayWritable
        }
}

public static class Temp2Reducer
extends Reducer<Text, ArrayWritable, Text, IntWritable>{
@Override public void reduce(Text key, Iterable<ArrayWritable> values, Context context ) 
            throws IOException, InterruptedException {

          int max = Integer.MIN_VALUE;
          int[] arr= new int[3];

          for (ArrayWritable val : values) {
              arr = (Int) val.get(); // Error: cannot cast Writable to int
              max = Math.max(max, arr[0]);
          }

          context.write( key, new IntWritable(max) );
        }

}

解决方案

ArrayWritable#get method returns an array of Writable.

You can't cast an array of Writable to int. What you can do is:

1. iterate over this array
2. cast each item (which will be of type Writable) of the array to IntWritable
3. use IntWritable#get method to get the int value.

for (ArrayWritable val: values) {
  for (Writable writable: val.get()) {                 // iterate
     IntWritable intWritable = (IntWritable)writable;  // cast
     int value = intWritable.get();                    // get
     // do your thing with int value
  }
}

这篇关于读取包装在Hadoop ArrayWritable中的值的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！