hadoop新手.分布式缓存

Question

我有一个映射器

public class BuildGraph{
   public void config(JobConf job){ <==this block doesnt seems to be exexcuting at all :(
    super.configure(job);
    this.currentId = job.getInt("currentId",0);
    if (this.currentId!=0){ 
    // I call a method from differnt class to build a distributed cache
    }
   }
  public void map(....){
....  
}

}

现在是称为..的主代码

public void run( String params,curId){
 JobConf conf = new JobConf(classname.class);
 conf.setInt("currentId",299); <--note this i am setting the value here
 conf.setMapperClass(BuildGraph.class);
 //.... 
  JobClient.runJob(conf);
}

但问题是代码中的配置方法没有执行，虽然"currentId"在主循环中返回299，但它在映射器类中根本没有设置。我做错了什么

链接到完整代码http://pastebin.com/DRPXUm62

Answer 1

看起来你没有使用正确的合同，因为你没有扩展MapReduceBase，也没有实现Mapper。该方法也应称为configure，而不是config。试试这样做:

public class BuildGraph extends MapReduceBase implements Mapper<K, V, K, V> {
    public void configure(JobConf job) {
        // this will get called once at the beginning of the task
    }
    public void map(...) {
        ...
    }
}