你能告诉我newAPIHadoopRDD的Maven依赖吗?

Question

你能告诉我newAPIHadoopRDD的Maven依赖吗？我的代码是：

JavaPairRDD<ImmutableBytesWritable, Result> hBaseRDD = ctx.newAPIHadoopRDD(
                        conf,
                        TextInputFormat.class,,
                        org.apache.hadoop.hbase.io.ImmutableBytesWritable.class,
                        org.apache.hadoop.hbase.client.Result.class);

提前感谢您的帮助。

Answer 1

您

正在使用 TextInputFormat。它必须是 TableInputFormat。以及您使用的conf，是SparkConf还是org.apache.hadoop.conf.Configuration？

Maven Dependencies。

    <dependency>
        <groupId>org.apache.hbase</groupId>
        <artifactId>hbase-client</artifactId>
        <version>${hbase.version}</version>
    </dependency>
    <dependency>
        <groupId>org.apache.hbase</groupId>
        <artifactId>hbase-common</artifactId>
        <version>${hbase.version}</version>
    </dependency>
    <dependency>
        <groupId>org.apache.hbase</groupId>
        <artifactId>hbase-server</artifactId>
        <version>${hbase.version}</version>
    </dependency>
    <dependency>
        <groupId>org.apache.hbase</groupId>
        <artifactId>hbase-hadoop-compat</artifactId>
        <version>${hbase.version}</version>
    </dependency>
    <dependency>
        <groupId>org.apache.hbase</groupId>
        <artifactId>hbase-protocol</artifactId>
        <version>${hbase.version}</version>
    </dependency>
    <dependency>
        <groupId>org.apache.hbase</groupId>
        <artifactId>hbase-hadoop2-compat</artifactId>
        <version>${hbase.version}</version>
    </dependency>
    <!-- HBase dependencies Start -->

法典：

Configuration hbaseConf = HBaseConfiguration.create();
hbaseConf.set("hbase.zookeeper.quorum", "localhost");
hbaseConf.set(TableInputFormat.INPUT_TABLE, "table1");
JavaPairRDD<ImmutableBytesWritable, Result> routerRDD = sc.newAPIHadoopRDD(hbaseConf, TableInputFormat.class, ImmutableBytesWritable.class, Result.class);