我一直在做一个MapReduce程序,我遇到了一个障碍,需要一些帮助。我有一个运行3个作业的程序(作业#2在for循环中运行5次),似乎我的一些映射器和reducer没有正确定义。在编译时,我一直得到"方法不覆盖或实现超类型的方法"错误。
下面是我的程序的基本结构: Job 1:
FirstMapper
没有减速机
Job 2:
第二Mapper
第一减速机
Job 3:
最终映射器最后减速器
我是这样定义映射器和还原器的:
public static class FirstMapper extends Mapper<Object, Text, LongWritable, Vertex> {
@Override
public void map(Object key, Text value, Context context) throws IOException, InterruptedException {}
public static class SecondMapper extends Mapper<LongWritable, Vertex, LongWritable, Vertex> {
@Override
public void map(long key, Vertex value, Context context) throws IOException, InterruptedException {}
public static class FirstReducer extends Reducer<LongWritable, Vertex, LongWritable, Vertex> {
@Override
public void reduce(Long key, Iterable<Vertex> values, Context context) throws IOException, InterruptedException {}
public static class FinalMapper extends Mapper<LongWritable, Vertex, LongWritable, LongWritable> {
@Override
public void map(Long key, Vertex value, Context context) throws IOException, InterruptedException {}
public static class FinalReducer extends Reducer<LongWritable, LongWritable, LongWritable, LongWritable> {
@Override
public void reduce(Long key, Iterable<LongWritable> values, Context context) throws IOException, InterruptedException {}
FirstMapper似乎是好的,因为它不会导致错误,但其他4做,我不知道为什么,因为方法签名看起来正确。我最初认为也许是因为我的自定义顶点类,但它实现了Writable所以我不确定为什么这会导致一个问题。任何和所有的帮助将非常感激。由于
您的密钥需要实现WritableComparable
接口并匹配您在Mapper
和Reducer
类签名中指定的类型。例如在第二个句子中,你有:
Mapper<LongWritable, Vertex, LongWritable, Vertex>
但是在map
方法中,您使用long
。这需要匹配您在签名中指定的类型,即LongWritable
。所以第二个映射的参数应该是这样的:
map(LongWritable key, Vertex value, Context context)
最后四个类都有这个问题