如何在Reducer中访问映射计数器值？ [英] how to access Mapper Counter value in a Reducer?

问题描述

我想访问reducer中的myCounter.my值：

  public static class Map扩展Mapper< LongWritable，Text， ImmutableBytesWritable，ImmutableBytesWritable> 
 {
 public static enum myCounter {my}; 
 
 @Override 
 public void map（LongWritable key，Text value，Context context）
 {
 context.getCounter（myCounter.my）.increment（1）; 
 context.write（new ImmutableBytesWritable（），new ImmutableBytesWritable（））; 
 
 
 
 
 public static class Reduce extends Reducer< ImmutableBytesWritable，ImmutableBytesWritable，Text，Text> 
 {
 @Override 
 public void reduce（ImmutableBytesWritable key，Iterable< ImmutableBytesWritable> result，Context context）
 {
 
} 
} b 
 
 
 $ b 
 mappers-counter-from-a-reducer？rq = 1>从reducer访问mapper的计数器（给出旧API） 
如何使它适用于新的API？  
 
 
 或 
 
 
我想知道mapper输出的总数量吗？有没有更好的方法？ （我无法在Reducer中访问计数器：
 
 
 组名称 - > org.apache.hadoop.mapred.Task $ Counter Counter Name - > MAP_OUTPUT_RECORDS ）
 
 
 
感谢 
 
解决方案
 
 配置conf = context.getConfiguration（ ）; 
群集群集=新群集（conf）; 
 Job currentJob = cluster.getJob（context.getJobID（））; 
 long val = currentJob.getCounters（）。findCounter（myCounter.my）.getValue（）; 
  
 
I want to acces the myCounter.my value in reducer :
public static class Map extends Mapper<LongWritable, Text, ImmutableBytesWritable, ImmutableBytesWritable>
{
    public static enum myCounter{my};

    @Override
    public void map(LongWritable key, Text value, Context context) 
    {
        context.getCounter(myCounter.my).increment(1);
        context.write( new ImmutableBytesWritable ( ),new ImmutableBytesWritable() );
    }
}


public static class Reduce extends Reducer<ImmutableBytesWritable, ImmutableBytesWritable, Text, Text>
{
    @Override
    public void reduce(ImmutableBytesWritable key,Iterable<ImmutableBytesWritable> result,Context context)
    {

    }
}
Accessing a mapper's counter from a reducer(for old API is given )
how to make it work for new API ? 

Or 

I want to know the total number of mapper output ? Is there any better way ? (i am not able to access counter in Reducer:

Group Name->org.apache.hadoop.mapred.Task$Counter  Counter Name->MAP_OUTPUT_RECORDS)

Thanks 
解决方案
You to make it work for new API by accessing the counters via job object.
Configuration conf = context.getConfiguration();
Cluster cluster = new Cluster(conf);
Job currentJob = cluster.getJob(context.getJobID());
long val=currentJob.getCounters().findCounter(myCounter.my).getValue();


                        
这篇关于如何在Reducer中访问映射计数器值？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！