以下命令在Linux上有效,但在Windows上失败。在运行命令之前,我使用kinit获取有效的Kerberos票证。
curl -v -i --negotiate -u : -b ~/cookiejar.txt -c ~/cookiejar.txt "http://<HDP-Cluster>:50070/webhdfs/v1/?op=LISTSTATUS"
预期结果(从Linux输出(:
> {"FileStatuses":{"FileStatus":[
> {"accessTime":0,"blockSize":0,"childrenNum":17,"fileId":18894,"group":"hadoop","length":0,"modificationTime":1589364305719,"owner":"yarn","pathSuffix":"app-logs","permission":"1777","replication":0,"storagePolicy":0,"type":"DIRECTORY"},
> {"accessTime":0,"blockSize":0,"childrenNum":3,"fileId":16432,"group":"hdfs","length":0,"modificationTime":1566323580684,"owner":"hdfs","pathSuffix":"apps","permission":"755","replication":0,"storagePolicy":0,"type":"DIRECTORY"},
> {"accessTime":0,"blockSize":0,"childrenNum":1,"fileId":16419,"group":"hdfs","length":0,"modificationTime":1566323394432,"owner":"hdfs","pathSuffix":"atsv2","permission":"755","replication":0,"storagePolicy":0,"type":"DIRECTORY"},
> {"accessTime":0,"blockSize":0,"childrenNum":0,"fileId":77127,"group":"hdfs","length":0,"modificationTime":1566983700653,"owner":"hdfs","pathSuffix":"benchmarks","permission":"755","replication":0,"storagePolicy":0,"type":"DIRECTORY"}
> ]}}
Windows输出:
* Connected to <SERVER> (<IP>) port 50070 (#0)
* Server auth using Negotiate with user ''
> GET /webhdfs/v1/?op=LISTSTATUS HTTP/1.1
> Host: <SERVER>:50070
> Authorization: Negotiate TlRMTVNTUAABAAAAt4II4gAAAAAAAAAAAAAAAAAAAAAKAGNFAAAADw==
> User-Agent: curl/7.70.0
> Accept: */*
> Cookie: hadoop.auth=
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 403 java.lang.IllegalArgumentException
< Date: Wed, 15 Jul 2020 09:32:34 GMT
< Date: Wed, 15 Jul 2020 09:32:34 GMT
< Pragma: no-cache
< X-FRAME-OPTIONS: SAMEORIGIN
* Replaced cookie hadoop.auth="" for domain <SERVER.Full.Domain>, path /, expire 0
< Set-Cookie: hadoop.auth=; Path=/; HttpOnly
< Cache-Control: must-revalidate,no-cache,no-store
< Content-Type: text/html;charset=iso-8859-1
< Content-Length: 293
<
{ [293 bytes data]
100 293 100 293 0 0 3367 0 --:--:-- --:--:-- --:--:-- 3367HTTP/1.1 403 java.lang.IllegalArgumentException
Date: Wed, 15 Jul 2020 09:32:34 GMT
Date: Wed, 15 Jul 2020 09:32:34 GMT
Pragma: no-cache
X-FRAME-OPTIONS: SAMEORIGIN
Set-Cookie: hadoop.auth=; Path=/; HttpOnly
Cache-Control: must-revalidate,no-cache,no-store
Content-Type: text/html;charset=iso-8859-1
Content-Length: 293
<html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 403 java.lang.IllegalArgumentException</title>
</head>
<body><h2>HTTP ERROR 403</h2>
<p>Problem accessing /webhdfs/v1/. Reason:
<pre> java.lang.IllegalArgumentException</pre></p>
</body>
</html>
* Connection #0 to host <server> left intact
curl版本窗口:
curl -V
curl 7.70.0 (x86_64-w64-mingw32) libcurl/7.70.0 OpenSSL/1.1.1g (Schannel) zlib/1.2.11 libidn2/2.3.0 libssh2/1.9.0 nghttp2/1.40.0
Release-Date: 2020-04-29
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smtp smtps telnet tftp
Features: AsynchDNS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz Metalink MultiSSL NTLM SPNEGO SSL SSPI TLS-SRP
curl版本Linux:
curl -V
curl 7.29.0 (x86_64-redhat-linux-gnu) libcurl/7.29.0 NSS/3.36 zlib/1.2.7 libidn/1.28 libssh2/1.4.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smtp smtps telnet tftp
Features: AsynchDNS GSS-Negotiate IDN IPv6 Largefile NTLM NTLM_WB SSL libz unix-sockets
我在上找到了一个关于如何在Linux上安装的详细描述。当使用--与curl协商时,是否需要keytab文件?,但是我找不到用于Windows的GSS-API,用于Windows的SSPI与用于Linux的GSS-API相同吗?
通过以下两个方面,我找到了答案:
在使用--convergent with curl时,是否需要keytab文件?
注意事项:编译Windows版本的curl是为了支持Microsoft SSPI库,而不是Hadoop REST服务所需的GSSAPI库
:(
在没有SPnego 的情况下访问kerberos安全WebHDFS
显示了一个Java代码,您在其中获得了一个委派令牌并将其传递给WebHDFSRestAPI。这种方式你不需要使用——谈判。