如何测试将键作为空发出键的映射器，即 context.write(null， <somevalue>);

Question

我有一个mapreduce程序，只有mapper，没有reducer集。我想测试一下。我有低于测试代码的

@Test
    public void testMapper() throws IOException {
      mapDriver.withInput(new LongWritable(0l), new Text(
              "af00bac654249b9d27982f19064338f4,54.0258822077885,-1.56832133466378,20121022,105507,026542913532,2093,87"));
      mapDriver.withOutput(null, [some value]);
      mapDriver.runTest();
    }

with调用mapDriver.withOutput（null，[some-value]）；它在异常下方抛出的这条线

java.lang.NullPointerException网址：org.apache.hadop.mrunit.internal.io.Serialization.copy（Serialization.java:58）网址：org.apache.hadop.mrunit.internal.io.Serialization.copy（Serialization.java:91）网址：org.apache.hadop.mrunit.internal.io.Serialization.copyWithConf（Serialization.java:104）

Note: Mapper Generic Signature Mapper<LongWritable, Text, Void, GenericRecord>

有人能告诉我如何为发出null的映射器编写测试场景吗？

如果我做Nullwritable.get，那么我会得到如下异常java.lang.NullPointerException网址：org.apache.hadop.mrunit.internal.io.Serialization.copy（Serialization.java:73）网址：org.apache.hadop.mrunit.internal.io.Serialization.copy（Serialization.java:91）网址：org.apache.hadop.mrunit.internal.io.Serialization.copyWithConf（Serialization.java:104）网址：org.apache.hadop.mrunit.TestDriver.copy（TestDriver.java:608）网址：org.apache.hadop.mrunit.TestDriver.copyPair（TestDriver.java:612）网址：org.apache.hadop.mrunit.TestDriver.addOutput（TestDriver.java:118）网址：org.apache.hadop.mrunit.TestDriver.withOutput（TestDriver.java:138）网址：com.gfk.gxl.etl.common.ExtractCSVTest.testMapper（ExtractCSVT est.java:73）在sun.reflect.NativeMethodAccessorImpl.invoke0（本机方法）

它看起来更像是序列化中带有Avro NullPointerException的MRUnit然而，答案并不能解决我的问题

 with few more research i have below update
    class org.apache.avro.generic.GenericData$Record is not able to get serializer and deserializer
    in org.apache.hadoop.mrunit.internal.io.Serialization and both are coming as null which is causing the null pointer exception

 From API code snippet  for org.apache.hadoop.mrunit.internal.io.Serialization starting at line      no 61 to 70
  try {
      serializer = (Serializer<Object>) serializationFactory
          .getSerializer(clazz);
      deserializer = (Deserializer<Object>) serializationFactory
          .getDeserializer(clazz);
    } catch (NullPointerException e) {
      throw new IllegalStateException(
          "No applicable class implementing Serialization in conf at io.serializations for "
              + orig.getClass(), e);
    }
above method serializer  deserializer  are coming null . do we have some way to avoid it

Answer 1

使用NullWritable.get（）方法。希望这能有所帮助。

Answer 2

不幸的是，虽然Hadoop可以接受空键，但您现在不能在MRUnit中使用空键，MRUnit团队计划在未来支持空键，请参阅此处允许空键和值作为输出，预期输出