在reudcer类中使用全局变量 [英] Use global variable in reudcer class

问题描述

我需要在我的mapreduce程序中使用全局变量，如何在以下代码中设置它，并在缩减器中使用全局变量。

pre $ code > public class tfidf
{
public static tfidfMap ..............
{
}
public static tfidfReduce ... ..........
{
}
public static void main（String args []）
{
Configuration conf = new Configuration（） ;
conf.set（，）;
}

}

解决方案

模板代码可能看起来像这样（Reducer没有显示，但是是相同的主体）

  import java.io.IOException; 
 
导入org.apache.hadoop.conf.Configuration; 
导入org.apache.hadoop.conf.Configured; 
 import org.apache.hadoop.io.LongWritable; 
 import org.apache.hadoop.io.Text; 
 import org.apache.hadoop.mapreduce.Job; 
 import org.apache.hadoop.mapreduce.Mapper; 
 import org.apache.hadoop.mapreduce.Mapper.Context; 
 import org.apache.hadoop.util.Tool; 
 import org.apache.hadoop.util.ToolRunner; 
 
 public class ToolExample extends Configured implements Tool {
 $ b @Override 
 public int run（String [] args）throws Exception {
 Job job = new工作（getConf（））; 
配置conf = job.getConfiguration（）; 
 
 conf.set（strProp，value）; 
 conf.setInt（intProp，123）; 
 conf.setBoolean（boolProp，true）; 
 
 //这里的其余配置
 // .. 
 
 return job.waitForCompletion（true）？ 0：1; 
} 
 
 public static class MyMapper extends 
 Mapper< LongWritable，Text，LongWritable，Text> {
 private String strProp; 
 private int intProp; 
 private boolean boolProp; 
 
 @Override 
 protected void setup（Context context）抛出IOException，
 InterruptedException {
 Configuration conf = context.getConfiguration（）; 
 
 strProp = conf.get（strProp）; 
 intProp = conf.getInt（intProp，-1）; 
 boolProp = conf.getBoolean（boolProp，false）; 
 
 
 $ b $ public static void main（String args []）throws Exception {
 System.exit（ToolRunner.run（new ToolExample（），args） ）; 
} 
} 
  

I need to use global variable in my mapreduce program how to set it in following code and use global variable in reducer.

public class tfidf
{
  public static tfidfMap..............
  {
  }
  public static tfidfReduce.............
  {
  }
  public static void main(String args[])
  {
       Configuration conf=new Configuration();
       conf.set("","");
  } 

}

解决方案

Template code could look something like this (Reducer not shown but is the same principal)

import java.io.IOException;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.conf.Configured;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.Mapper;
import org.apache.hadoop.mapreduce.Mapper.Context;
import org.apache.hadoop.util.Tool;
import org.apache.hadoop.util.ToolRunner;

public class ToolExample extends Configured implements Tool {

    @Override
    public int run(String[] args) throws Exception {
        Job job = new Job(getConf());
        Configuration conf = job.getConfiguration();

        conf.set("strProp", "value");
        conf.setInt("intProp", 123);
        conf.setBoolean("boolProp", true);

        // rest of your config here
        // ..

        return job.waitForCompletion(true) ? 0 : 1;
    }

    public static class MyMapper extends
            Mapper<LongWritable, Text, LongWritable, Text> {
        private String strProp;
        private int intProp;
        private boolean boolProp;

        @Override
        protected void setup(Context context) throws IOException,
                InterruptedException {
            Configuration conf = context.getConfiguration();

            strProp = conf.get("strProp");
            intProp = conf.getInt("intProp", -1);
            boolProp = conf.getBoolean("boolProp", false);
        }
    }

    public static void main(String args[]) throws Exception {
        System.exit(ToolRunner.run(new ToolExample(), args));
    }
}

这篇关于在reudcer类中使用全局变量的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！