Hadoop - 如何发出 TwoDArrayWriable

Question

我正在从映射器发出TwoDArrayWwriteable。我已经实现了扩展TwoDArrayWwriteable的类，以便制作默认构造函数。但是当我尝试发出它时，它给了我以下异常：

Error: java.lang.RuntimeException: java.lang.InstantiationException: org.apache.hadoop.io.TwoDArrayWritable
        at org.apache.hadoop.io.TwoDArrayWritable.readFields(TwoDArrayWritable.java:75)
        at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:71)

如何发出 TwoDArrayWwriteable？我需要一些帮助。

下面是扩展 TwoDArrayWwrite 的类：

public class TwoDArrayWritables extends TwoDArrayWritable
{
    public TwoDArrayWritables() {
        super(TwoDArrayWritable.class);
    }

    public TwoDArrayWritables(Class valueClass) {
        super(valueClass);
        // TODO Auto-generated constructor stub
    }

这是映射器：

public class JaccardMapper extends Mapper<LongWritable, Text, IntTextPair, TwoDArrayWritables> {
    Hashtable movieInfo = new Hashtable<String, String>();
    String[] genres, actors, entities;
    String[] attributes = new String[] {"genre", "actors", "directors", "country", "year", "ratings"};
    double p,q,r,s;
    double result = 0.0;
    String input[] = null;
    Set<String> keys;
    TwoDArrayWritables array2d = new TwoDArrayWritables();
    //TwoDArrayWritable array2d = new TwoDArrayWritable(IntWritable.class);
    IntWritable[][] jaccard = new IntWritable[2][];
    //int[][] jaccard = new int[2][];

    public void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException
    {
        p = 0;
        q = 0;
        r = 0;
        s = 0;
        input = value.toString().toLowerCase().split(",");
        keys = movieInfo.keySet();

        //the jaccards 2d array column length depends on the user input best case is 6 but the worst case depends on the sub attributes count like more than one actor/director/genre/country.  
        int columnlength = input[1].split("\|").length + input[2].split("\|").length + input[3].split("\|").length + input[4].split("\|").length + 2;
        jaccard = new IntWritable[2][columnlength];
        for (int i = 0; i < jaccard.length; i++)
        {
            for (int j = 0; j < jaccard[i].length; j++)
            {
                jaccard[i][j] = new IntWritable(0);
            }
        }
        if (input.length > 0)
        {
            //iterate through the dataset in cache
            for(String keyy : keys)
            {
                //iterate to user's input attributes
                for (int attribute = 1; attribute < attributes.length; attribute++)
                {
                    if (!input[attribute].equals("-")) 
                    {
                        entities = input[attribute].toLowerCase().split("\|");
                        int subattributecount = 0;
                        for(String entity : entities)
                        {
                                subattributecount += 1;
                        }
                    }
                }
                IntTextPair pair = new IntTextPair(Integer.parseInt(input[0].toString()), movieInfo.get(keyy).toString());
                array2d.set(jaccard);
                                context.write(pair, array2d);
            }

        }
}
}

这是减速器：公共类 JaccardReducer 扩展 Reducer {

double p,q,r,s;
double result = 0.0;
//IntWritable[][] jaccard = null;
IntWritable[][] jaccard;
int temp1 = 0, temp2 = 0;
//Jaccard distance 1.0 emplies that the user's criteria doesn't meet at all
public static final double nonacceptvalue = 1.0;

    public void reduce(IntTextPair key, Iterable<TwoDArrayWritables> values, Context context) throws IOException, InterruptedException
    {
        IntDoublePair pair = new IntDoublePair(key.getFirst().get(), Double.parseDouble("110.00"));
        for (TwoDArrayWritable value : values)
        {
            if (value != null)
            {
                context.write(pair, new Text("Is not null"));
            }
            else
            {
                context.write(pair, new Text("Is null"));
            }
        }
}
}

Answer 1

在默认构造函数（在 super）中指定的类应该是值的类，而不是整个类本身的类。 所以你可能想要：

public TwoDArrayWritables() {
    super(IntWritable.class);
}