SparkSQL的Airbnb超集数据量数据量配置

Question

我正在使用Spark 1.6.2（来自DataStax Enterprise Edition => dse 5.0.4）和Python 2.7

当我给出

时

from impala.dbapi import connect    
Conn = connect (host = '172.31.12.201', port = 7077, user = 'xxxx', password = 'xxxx1111', database = 'test_database', auth_mechanism = 'PLAIN') 

它只是悬挂而永远不会出来
Spark Master在港口7077上的172.31.12.201

我在超集中的配置如下

SQLAlchemy URI => impala://172.31.12.201:7077/test_database
Extra => {
"metadata_params": {},
"engine_params": {"connect_args": {"user": "xxxx", "password": "xxxx1111"}} }

Answer 1

我必须启动DSE Spark Thrift Server，如下

dse -u <username> -p <password> spark-sql-thriftserver start

这将在端口10000