如何使用 impyla 连接到黑斑羚或使用 pyhive 连接到蜂巢?



我正在尝试使用以下代码使用 impyla 连接到黑斑羚:

from impala.dbapi import connect
conn = connect(host='host_name.com', port=21050, user='usr', password='pass', use_ssl=True, auth_mechanism='LDAP')
cursor = conn.cursor()
cursor.execute('SHOW DATABASES')
cursor.fetchall()

根据文档,该库需要在 0.2.1 版中thrift_sasl,但我无法安装它,因为它显示此错误

Collecting thrift_sasl==0.2.1
Using cached https://files.pythonhosted.org/packages/80/36/16dfe92d32df63cc2b7b7be8d0e4a736617b7e52daaa7f83ae386a89d179/thrift_sasl-0.2.1.tar.gz
Collecting sasl>=0.2.1 (from thrift_sasl==0.2.1)
Using cached https://files.pythonhosted.org/packages/8e/2c/45dae93d666aea8492678499e0999269b4e55f1829b1e4de5b8204706ad9/sasl-0.2.1.tar.gz
Collecting thriftpy (from thrift_sasl==0.2.1)
Using cached https://files.pythonhosted.org/packages/f4/19/cca118cf7d2087310dbc8bd70dc7df0c1320f2652873a93d06d7ba356d4a/thriftpy-0.3.9.tar.gz
Requirement already satisfied: six in c:userspsowaappdatalocalprogramspythonpython36libsite-packages (from sasl>=0.2.1->thrift_sasl==0.2.1) (1.12.0)
Requirement already satisfied: ply<4.0,>=3.4 in c:userspsowaappdatalocalprogramspythonpython36libsite-packages (from thriftpy->thrift_sasl==0.2.1) (3.11)
Installing collected packages: sasl, thriftpy, thrift-sasl
Running setup.py install for sasl ... error
ERROR: Command errored out with exit status 1:
command: 'c:userspsowaappdatalocalprogramspythonpython36python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\psowa\AppData\Local\Temp\pip-install-y42wej4x\sasl\setup.py'"'"'; __file__='"'"'C:\Users\psowa\AppData\Local\Temp\pip-install-y42wej4x\sasl\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'rn'"'"', '"'"'n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:UserspsowaAppDataLocalTemppip-record-rjwkcc76install-record.txt' --single-version-externally-managed --compile
cwd: C:UserspsowaAppDataLocalTemppip-install-y42wej4xsasl
Complete output (27 lines):
running install
running build
running build_py
creating build
creating buildlib.win-amd64-3.6
creating buildlib.win-amd64-3.6sasl
copying sasl__init__.py -> buildlib.win-amd64-3.6sasl
running egg_info
writing sasl.egg-infoPKG-INFO
writing dependency_links to sasl.egg-infodependency_links.txt
writing requirements to sasl.egg-inforequires.txt
writing top-level names to sasl.egg-infotop_level.txt
reading manifest file 'sasl.egg-infoSOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'sasl.egg-infoSOURCES.txt'
copying saslsaslwrapper.cpp -> buildlib.win-amd64-3.6sasl
copying saslsaslwrapper.h -> buildlib.win-amd64-3.6sasl
copying saslsaslwrapper.pyx -> buildlib.win-amd64-3.6sasl
running build_ext
building 'sasl.saslwrapper' extension
creating buildtemp.win-amd64-3.6
creating buildtemp.win-amd64-3.6Release
creating buildtemp.win-amd64-3.6Releasesasl
C:Program Files (x86)Microsoft Visual Studio2017CommunityVCToolsMSVC14.16.27023binHostX86x64cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -Isasl -Ic:userspsowaappdatalocalprogramspythonpython36include -Ic:userspsowaappdatalocalprogramspythonpython36include "-IC:Program Files (x86)Microsoft Visual Studio2017CommunityVCToolsMSVC14.16.27023ATLMFCinclude" "-IC:Program Files (x86)Microsoft Visual Studio2017CommunityVCToolsMSVC14.16.27023include" "-IC:Program Files (x86)Windows Kits10include10.0.17763.0ucrt" "-IC:Program Files (x86)Windows Kits10include10.0.17763.0shared" "-IC:Program Files (x86)Windows Kits10include10.0.17763.0um" "-IC:Program Files (x86)Windows Kits10include10.0.17763.0winrt" "-IC:Program Files (x86)Windows Kits10include10.0.17763.0cppwinrt" /EHsc /Tpsasl/saslwrapper.cpp /Fobuildtemp.win-amd64-3.6Releasesasl/saslwrapper.obj
saslwrapper.cpp
c:userspsowaappdatalocaltemppip-install-y42wej4xsaslsaslsaslwrapper.h(22): fatal error C1083: Cannot open include file: 'sasl/sasl.h': No such file or directory
error: command 'C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\bin\HostX86\x64\cl.exe' failed with exit status 2
----------------------------------------
ERROR: Command errored out with exit status 1: 'c:userspsowaappdatalocalprogramspythonpython36python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\psowa\AppData\Local\Temp\pip-install-y42wej4x\sasl\setup.py'"'"'; __file__='"'"'C:\Users\psowa\AppData\Local\Temp\pip-install-y42wej4x\sasl\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'rn'"'"', '"'"'n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:UserspsowaAppDataLocalTemppip-record-rjwkcc76install-record.txt' --single-version-externally-managed --compile Check the logs for full command output.

当我安装最新版本的 Jupyter 时thrift_sasl发生此错误:

AttributeError: 'TSSLSocket' object has no attribute 'isOpen'

我还尝试使用以下代码通过 pyhive 进行连接:

from pyhive import hive
host_name = "host_name.com"
port = 10000
user = "usr"
password = "pass"
def hiveconnection(host_name, port, user,password):
conn = hive.Connection(host=host_name, port=port, username=user, password=password, auth='LDAP')
cur = conn.cursor()
cur.execute('SHOW DATABASES')
result = cur.fetchall()
return result
output = hiveconnection(host_name, port, user,password)
print(output)

它希望我安装 sasl,但是当我尝试这样做时,它显示:

Collecting sasl
Using cached https://files.pythonhosted.org/packages/8e/2c/45dae93d666aea8492678499e0999269b4e55f1829b1e4de5b8204706ad9/sasl-0.2.1.tar.gz
Requirement already satisfied: six in c:userspsowaappdatalocalprogramspythonpython36libsite-packages (from sasl) (1.12.0)
Installing collected packages: sasl
Running setup.py install for sasl ... error
ERROR: Command errored out with exit status 1:
command: 'c:userspsowaappdatalocalprogramspythonpython36python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\psowa\AppData\Local\Temp\pip-install-9rn_a9g0\sasl\setup.py'"'"'; __file__='"'"'C:\Users\psowa\AppData\Local\Temp\pip-install-9rn_a9g0\sasl\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'rn'"'"', '"'"'n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:UserspsowaAppDataLocalTemppip-record-sedxucbainstall-record.txt' --single-version-externally-managed --compile
cwd: C:UserspsowaAppDataLocalTemppip-install-9rn_a9g0sasl
Complete output (27 lines):
running install
running build
running build_py
creating build
creating buildlib.win-amd64-3.6
creating buildlib.win-amd64-3.6sasl
copying sasl__init__.py -> buildlib.win-amd64-3.6sasl
running egg_info
writing sasl.egg-infoPKG-INFO
writing dependency_links to sasl.egg-infodependency_links.txt
writing requirements to sasl.egg-inforequires.txt
writing top-level names to sasl.egg-infotop_level.txt
reading manifest file 'sasl.egg-infoSOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'sasl.egg-infoSOURCES.txt'
copying saslsaslwrapper.cpp -> buildlib.win-amd64-3.6sasl
copying saslsaslwrapper.h -> buildlib.win-amd64-3.6sasl
copying saslsaslwrapper.pyx -> buildlib.win-amd64-3.6sasl
running build_ext
building 'sasl.saslwrapper' extension
creating buildtemp.win-amd64-3.6
creating buildtemp.win-amd64-3.6Release
creating buildtemp.win-amd64-3.6Releasesasl
C:Program Files (x86)Microsoft Visual Studio2017CommunityVCToolsMSVC14.16.27023binHostX86x64cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -Isasl -Ic:userspsowaappdatalocalprogramspythonpython36include -Ic:userspsowaappdatalocalprogramspythonpython36include "-IC:Program Files (x86)Microsoft Visual Studio2017CommunityVCToolsMSVC14.16.27023ATLMFCinclude" "-IC:Program Files (x86)Microsoft Visual Studio2017CommunityVCToolsMSVC14.16.27023include" "-IC:Program Files (x86)Windows Kits10include10.0.17763.0ucrt" "-IC:Program Files (x86)Windows Kits10include10.0.17763.0shared" "-IC:Program Files (x86)Windows Kits10include10.0.17763.0um" "-IC:Program Files (x86)Windows Kits10include10.0.17763.0winrt" "-IC:Program Files (x86)Windows Kits10include10.0.17763.0cppwinrt" /EHsc /Tpsasl/saslwrapper.cpp /Fobuildtemp.win-amd64-3.6Releasesasl/saslwrapper.obj
saslwrapper.cpp
c:userspsowaappdatalocaltemppip-install-9rn_a9g0saslsaslsaslwrapper.h(22): fatal error C1083: Cannot open include file: 'sasl/sasl.h': No such file or directory
error: command 'C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\bin\HostX86\x64\cl.exe' failed with exit status 2
----------------------------------------
ERROR: Command errored out with exit status 1: 'c:userspsowaappdatalocalprogramspythonpython36python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\psowa\AppData\Local\Temp\pip-install-9rn_a9g0\sasl\setup.py'"'"'; __file__='"'"'C:\Users\psowa\AppData\Local\Temp\pip-install-9rn_a9g0\sasl\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'rn'"'"', '"'"'n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:UserspsowaAppDataLocalTemppip-record-sedxucbainstall-record.txt' --single-version-externally-managed --compile Check the logs for full command output.

有什么想法吗?

在 2.7 版中使用 python 修复了此问题。我认为兼容性存在问题。

最新更新