以下是我的蜂巢/conf/conf/hive-site.xml:
<configuration>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://127.0.0.1/metastore?createDatabaseIfNotExist=true</value>
<description>metadata is stored in a MySQL server</description>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
<description>MySQL JDBC driver class</description>
</property>
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>hiveuser</value>
<description>user name for connecting to mysql server</description>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>hivepassword</value>
<description>password for connecting to mysql server</description>
</property>
</configuration>
我想使用Spark-HiveContext访问Hive现有数据库和表。因此,将下面添加到Hive/conf/conf/hive-site.xml:
<configuration>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://127.0.0.1/metastore?createDatabaseIfNotExist=true</value>
<description>metadata is stored in a MySQL server</description>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
<description>MySQL JDBC driver class</description>
</property>
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>hiveuser</value>
<description>user name for connecting to mysql server</description>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>hivepassword</value>
<description>password for connecting to mysql server</description>
</property>
<property>
<name>hive.metastore.uris</name>
<value>thrift://127.0.0.1:9083</value>
</property>
</configuration>
在编辑Hive-site.xml之后,如上图所示,蜂巢壳不起作用。请帮助我更新hive-site.xml正确的方法,并帮助我使用hivecontext在Spark-Shell上访问Hive Tables,如下所示:
val hc = new org.apache.spark.sql.hive.HiveContext(sc);
hc.setConf("hive.metastore.uris","thrift://127.0.0.1:9083");
val a = hc.sql("show databases");
a.show //should display all my hive databases.
请在这个问题上帮助我。
@chaithu您需要使用Hive -Service Metastore启动Hive Metastore,然后以这种方式启用使用Hivesupport的Sparksession
val spark= SparkSession
.builder()
.master("local")
.appName("HiveExample").config("hive.metastore.uris","thrift://hadoop-master:9083")
.enableHiveSupport()
.getOrCreate()