通过HFILE将数据加载到HBase中不起作用

Question

我编写了一个映射器，以通过HFILE从磁盘加载数据到HBASE，该程序成功运行，但是我的HBase表中没有加载数据，对此有任何想法吗？

这是我的Java程序：

protected void writeToHBaseViaHFile() throws Exception {
        try {
            System.out.println("In try...");
            Configuration conf = HBaseConfiguration.create();
            conf.set("hbase.zookeeper.quorum", "XXXX");
            Connection connection = ConnectionFactory.createConnection(conf);
            System.out.println("got connection");
            String inputPath = "/tmp/nuggets_from_Hive/part-00000";
            String outputPath = "/tmp/mytemp" + new Random().nextInt(1000);
            final TableName tableName = TableName.valueOf("steve1");
            System.out.println("got table steve1, outputPath = " + outputPath);
            // tag::SETUP[]
            Table table = connection.getTable(tableName);
            Job job = Job.getInstance(conf, "ConvertToHFiles");
            System.out.println("job is setup...");
            HFileOutputFormat2.configureIncrementalLoad(job, table,
                connection.getRegionLocator(tableName)); // <1>
            System.out.println("done configuring incremental load...");
            job.setInputFormatClass(TextInputFormat.class); // <2>
            job.setJarByClass(Importer.class); // <3>
            job.setMapperClass(LoadDataMapper.class); // <4>
            job.setMapOutputKeyClass(ImmutableBytesWritable.class); // <5>
            job.setMapOutputValueClass(KeyValue.class); // <6>
            FileInputFormat.setInputPaths(job, inputPath);
            HFileOutputFormat2.setOutputPath(job, new org.apache.hadoop.fs.Path(outputPath));
            System.out.println("Setup complete...");
            // end::SETUP[]
            if (!job.waitForCompletion(true)) {
                System.out.println("Failure");
            } else {
                System.out.println("Success");
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

这是我的映射类：

public class LoadDataMapper extends Mapper<LongWritable, Text, ImmutableBytesWritable, Cell> {
    public static final byte[] FAMILY = Bytes.toBytes("pd");
    public static final byte[] COL = Bytes.toBytes("bf");
    public static final ImmutableBytesWritable rowKey = new ImmutableBytesWritable();
    @Override
    protected void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException {
        String[] line = value.toString().split("t"); // <1>
        byte[] rowKeyBytes = Bytes.toBytes(line[0]);
        rowKey.set(rowKeyBytes);
        KeyValue kv = new KeyValue(rowKeyBytes, FAMILY, COL, Bytes.toBytes(line[1])); // <6>
        context.write (rowKey, kv); // <7>
        System.out.println("line[0] = " + line[0] + "tline[1] = " + line[1]);
    }
}

我已经在我的群集中创建了表steve1，但是在程序成功运行后获得了0行：

hbase(main):007:0> count 'steve1'
0 row(s) in 0.0100 seconds
=> 0

我尝试了什么：

我尝试像在"映射"类中添加打印消息，以查看它是否真正读取数据，但是打印输出从未在我的控制台中打印出来。我对如何调试这个不知所措。

任何想法都非常感谢！

Answer 1

这仅是为了创建HFILE，您仍然需要将HFile加载到表上。例如，您需要做类似：

的事情

LoadIncrementalHFiles loader = new LoadIncrementalHFiles(conf);
loader.doBulkLoad(new Path(outputPath), admin, hTable, regionLocator);