HBase上的Map Reduce错误，原因

Question

我有以下工作，还有许多其他类似的工作，像这个一样启动

String jobName = MR.class
            .getSimpleName();
    Configuration config = HBaseConfiguration.create();
    config.set(
            "hbase.master", HBASE_MASTER);
    config.set(
            "hbase.zookeeper.quorum", HBASE_ZOOKEEPERS);
    config.set(
            "hbase.zookeeper.property.clientPort", ZK_CLIENT_PORT);
    Job job = Job.getInstance(config, jobName);
    job.setJarByClass(MR.class);
    Scan scan = new Scan();
    //gets the raw band data cells
    scan.addColumn("gc".getBytes(), "s".getBytes());
    System.out.println("Job=" + jobName);
    System.out.println("tHBASE_MASTER=" + HBASE_MASTER);
    System.out.println(
            "tHBASE_ZOOKEEPERS=" + HBASE_ZOOKEEPERS);
    System.out.println(
            "tINPUT_TABLE=" + INPUT_TABLE);
    System.out.println(
            "tOUTPUT_TABLE= " + OUTPUT_TABLE);
    System.out.println(
            "tCACHING_LEVEL= " + CACHING_LEVEL);
    // TODO: make caching size configurable
    scan.setCaching(CACHING_LEVEL);
    scan.setCacheBlocks(
            false);
    // null for output key/value since we're not sending anything to reduce
    TableMapReduceUtil.initTableMapperJob(INPUT_TABLE, scan,
            MR.MapClass.class,
            null,
            null,
            job);
    TableMapReduceUtil.initTableReducerJob(
            "PIXELS", // output table
            null, // reducer class
            job);
    job.setNumReduceTasks(0);
    // at least one, adjust as required
    boolean b = job.waitForCompletion(true);

但我一直收到这个错误，甚至无法将数据读取到映射器中

Error: java.io.BufferedReader.lines()Ljava/util/stream/Stream;
14/09/22 22:11:13 INFO mapreduce.Job: Task Id : attempt_1410880772411_0045_m_000009_2, Status : FAILED
Error: java.io.BufferedReader.lines()Ljava/util/stream/Stream;
14/09/22 22:11:13 INFO mapreduce.Job: Task Id : attempt_1410880772411_0045_m_000018_2, Status : FAILED
Error: java.io.BufferedReader.lines()Ljava/util/stream/Stream;
14/09/22 22:11:13 INFO mapreduce.Job: Task Id : attempt_1410880772411_0045_m_000002_2, Status : FAILED

以前从未见过这种情况，已经使用HBase一段时间了。。。。这是我尝试过的：

I can scan the table via the hbase shell no problem in the same way this job does
I can use the java api to scan in the same way no problem
I built and rebuilt my jar thinking some files were corrupt... not the problem
I have at least 5 other jobs setup the same way
I disabled, enabled, compacted, rebuilt everything you can imagine with the table
I am using maven shade to uber jar
I am running HBase 0.98.1-cdh5.1.0

非常感谢任何帮助或想法。

Answer 1

看来我应该等一段时间再问这个问题。问题是，我实际上正在获取数据，但我的一个外部jar是用Java 8构建的，并且在Java 7的BufferedReader中不存在.lines（），我正在运行Java 7，因此出现了错误。与HBase或Map Reduce无关。