连接 Kerberos + SSL 启用 solr 在纱线下的火花作业中

我有启用了Kerberos和SSL的SOLR 6集群。当我使用CloudSolrClient的测试客户端连接到它时，它工作正常。但是在火花作业驱动程序中运行时相同的代码，我得到以下校验和失败错误。

我检查了所有提到的与校验和相关的问题，例如反向 dns 查找和添加 java 无限罐子，所有 yarn 节点中一切看起来都正确。我还可以验证它们是否正确，因为我的普通 java 客户端能够从所有服务器进行查询。

Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://10-0-0-64.securonix.com:18987/solr/test3: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 403 GSSException: Failure unspecified at GSS-API level (Mechanism level: Checksum failed)</title>
</head>
<body><h2>HTTP ERROR 403</h2>
<p>Problem accessing /solr/test3/select. Reason:
<pre>    GSSException: Failure unspecified at GSS-API level (Mechanism level: Checksum failed)</pre></p>
</body>
</html>
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:578)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:447)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:388)
... 18 more
LogType:stdout
Log Upload Time:Wed May 09 13:29:07 -0400 20

我正在添加我的工作协商堆栈跟踪和非工作堆栈跟踪，请让我知道我可以从哪个地方开始调试它。

277313:29:01,363 DEBUG DefaultClientConnectionOperator:176 - Connecting to 10-0-0-64.company.com:18987
278013:29:01,457 DEBUG RequestAddCookies:122 - CookieSpec selected: solr-portaware
287413:29:01,458 DEBUG RequestAuthCache:76 - Auth cache not set in the context
287513:29:01,459 DEBUG RequestTargetAuthentication:79 - Target auth state: UNCHALLENGED
287613:29:01,459 DEBUG RequestProxyAuthentication:88 - Proxy auth state: UNCHALLENGED
287613:29:01,459 DEBUG SystemDefaultHttpClient:684 - Attempt 1 to execute request
287613:29:01,459 DEBUG DefaultClientConnection:276 - Sending request: GET /solr/test3/select?q=*%3A*&_stateVer_=test3%3A46&wt=javabin&version=2 HTTP/1.1
287613:29:01,460 DEBUG wire:72 -  >> "GET /solr/test3/select?q=*%3A*&_stateVer_=test3%3A46&wt=javabin&version=2 HTTP/1.1[r][n]"
287713:29:01,460 DEBUG wire:72 -  >> "User-Agent: Solr[org.apache.solr.client.solrj.impl.HttpSolrClient] 1.0[r][n]"
287713:29:01,460 DEBUG wire:72 -  >> "Host: 10-0-0-64.company.com:18987[r][n]"
287713:29:01,461 DEBUG wire:72 -  >> "Connection: Keep-Alive[r][n]"
287813:29:01,461 DEBUG wire:72 -  >> "[r][n]"
287813:29:01,461 DEBUG headers:280 - >> GET /solr/test3/select?q=*%3A*&_stateVer_=test3%3A46&wt=javabin&version=2 HTTP/1.1
287813:29:01,461 DEBUG headers:283 - >> User-Agent: Solr[org.apache.solr.client.solrj.impl.HttpSolrClient] 1.0
287813:29:01,461 DEBUG headers:283 - >> Host: 10-0-0-64.company.com:18987
287813:29:01,461 DEBUG headers:283 - >> Connection: Keep-Alive
287813:29:01,463 DEBUG wire:72 -  << "HTTP/1.1 401 Authentication required[r][n]"
288013:29:01,464 DEBUG wire:72 -  << "WWW-Authenticate: Negotiate[r][n]"
288113:29:01,464 DEBUG wire:72 -  << "Set-Cookie: hadoop.auth=; Path=/; Domain=10-0-0-64.company.com:18987; Expires=Thu, 01-Jan-1970 00:00:00 GMT; Secure; HttpOnly[r][n]"
288113:29:01,464 DEBUG wire:72 -  << "Cache-Control: must-revalidate,no-cache,no-store[r][n]"
288113:29:01,464 DEBUG wire:72 -  << "Content-Type: text/html;charset=iso-8859-1[r][n]"
288113:29:01,465 DEBUG wire:72 -  << "Content-Length: 277[r][n]"
288213:29:01,465 DEBUG wire:72 -  << "[r][n]"
288213:29:01,465 DEBUG DefaultClientConnection:261 - Receiving response: HTTP/1.1 401 Authentication required
288213:29:01,465 DEBUG headers:264 - << HTTP/1.1 401 Authentication required
288213:29:01,465 DEBUG headers:267 - << WWW-Authenticate: Negotiate
288213:29:01,466 DEBUG headers:267 - << Set-Cookie: hadoop.auth=; Path=/; Domain=10-0-0-64.company.com:18987; Expires=Thu, 01-Jan-1970 00:00:00 GMT; Secure; HttpOnly
288313:29:01,466 DEBUG headers:267 - << Cache-Control: must-revalidate,no-cache,no-store
288313:29:01,466 DEBUG headers:267 - << Content-Type: text/html;charset=iso-8859-1
288313:29:01,466 DEBUG headers:267 - << Content-Length: 277
288313:29:01,472 DEBUG ResponseProcessCookies:118 - Cookie accepted [hadoop.auth="", version:0, domain:10-0-0-64.company.com:18987, path:/, expiry:Wed Dec 31 19:00:00 EST 1969]
288913:29:01,473 DEBUG SystemDefaultHttpClient:511 - Connection can be kept alive indefinitely
289013:29:01,473 DEBUG SystemDefaultHttpClient:77 - Authentication required
289013:29:01,473 DEBUG SystemDefaultHttpClient:107 - 10-0-0-64.company.com:18987 requested authentication
289013:29:01,474 DEBUG TargetAuthenticationStrategy:174 - Authentication schemes in the order of preference: [Negotiate, Kerberos, NTLM, Digest, Basic]
289113:29:01,476 DEBUG SPNegoScheme:266 - Received challenge '' from the auth server
289313:29:01,477 DEBUG TargetAuthenticationStrategy:203 - Challenge for Kerberos authentication scheme not available
289413:29:01,477 DEBUG TargetAuthenticationStrategy:203 - Challenge for NTLM authentication scheme not available
289413:29:01,477 DEBUG TargetAuthenticationStrategy:203 - Challenge for Digest authentication scheme not available
289413:29:01,477 DEBUG TargetAuthenticationStrategy:203 - Challenge for Basic authentication scheme not available
289413:29:01,477 DEBUG SystemDefaultHttpClient:157 - Selected authentication options: [NEGOTIATE]
289413:29:01,478 DEBUG wire:72 -  << "<html>[n]"
289513:29:01,478 DEBUG wire:72 -  << "<head>[n]"
289513:29:01,478 DEBUG wire:72 -  << "<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>[n]"
289513:29:01,478 DEBUG wire:72 -  << "<title>Error 401 Authentication required</title>[n]"
289513:29:01,478 DEBUG wire:72 -  << "</head>[n]"
289513:29:01,479 DEBUG wire:72 -  << "<body><h2>HTTP ERROR 401</h2>[n]"
289613:29:01,479 DEBUG wire:72 -  << "<p>Problem accessing /solr/test3/select. Reason:[n]"
289613:29:01,479 DEBUG wire:72 -  << "<pre>    Authentication required</pre></p>[n]"
289613:29:01,479 DEBUG wire:72 -  << "</body>[n]"
289613:29:01,479 DEBUG wire:72 -  << "</html>[n]"
289613:29:01,479 DEBUG RequestAddCookies:122 - CookieSpec selected: solr-portaware
289613:29:01,479 DEBUG RequestAuthCache:76 - Auth cache not set in the context
289613:29:01,479 DEBUG RequestTargetAuthentication:79 - Target auth state: CHALLENGED
289613:29:01,480 DEBUG RequestTargetAuthentication:79 - Generating response to an authentication challenge using Negotiate scheme

****************加工**********************************

107718:07:10,421 DEBUG wire:72 -  >> "Authorization: Negotiate YIIFCQYGKwYBBQUCoIIE/TCCBPmgDTALBgkqhkiG9xIBAgKhBAMCAfaiggTgBIIE3GCCBNgGCSqGSIb3EgECAgEAboIExzCCBMOgAwIBBaEDAgEOogcDBQAgAAAAo4IBU2GCAU8wggFLoAMCAQWhDxsNU0VDVVJPTklYLkNPTaIqMCigAwIBAKEhMB8bBEhUVFAbFzEwLTAtMC02NC5zZWN1cm9uaXguY29to4IBBTCCAQGgAwIBEqEDAgEBooH0BIHxKKe1e1rmBivsMS7EM7QlfH/1P9AqnSEuZF/A9o2FBwt6X010f7braFpqBT7mXTU28PiUYecdtxva39av/UUQbSupla64CvR75PatuwiQYKtvnEBHINhX4/lv25Mm6SR7YiFM9o4jbxYXL9k/1bS6UitipNxA7xAX1cLLE270/j/3M4BpL3TgohZR9RVpFrL5rmGTVnrcP6nTZ0YSsaQ4js8fbldYQCxBAyTeiIwb3HGmrl68P/crWaQqv/9dkDS0yuvW26nQoTIwUuvlsTmYAB/Pgopi1hobiTaiMN7z7BIjBXjNBYUPTc6Ab9qxmtaQfqSCA1UwggNRoAMCARKiggNIBIIDROkC2Wypm9nMm60R0rZG+MkcbLLIjyqfSoliOrfCov+0Qko2nxpwOcm1ckdtNgKRnYzzCi9gWTSd9ZIisMPynY/YgCfc4a10QqKER+NwdJLxEXiNfqBKQ81xsp6llZ/I+WMpuZtmH5o8RAtAmamt4G0O31jtBwrSGFMzsxVYGi1kZeN0RqB9TvHScbFXe/TSJWoXc3qL/RzDz519MXQ/MTwYE9klvf15zXgakuEMPJbx0jJ7sVvwRTuNJ+mpv37qwX8JAKMls0045MJlc0zl/39OSh9edjnfi+IIM0K5FJ/wGgm3Vlt9t/rlCuYLlybfB5a3wZ78d/KOhmZwtBQ5arN/vXqE/UQz9rzRsbByZ7kDksphd98TgY3uPaq+SA1MBG5NrawLXxNpoxZZbgob6n6FLzesXQ2dnVMYuGLKoB/VZigAOqMR2wgLNCcNzI+dTvI7gTWf/ao/XfT5EFCd17kRH1XTvhaI6A1zULaGMNxzcoZJDlHZ0AHAnsUj4X0u4xXtVX3LaFKGkXtQNGQEsCtnQbygphl/rrgH06Mv3duhZiNqpb9TVpfbyJ6NoWzX5O4ud+meyAW4rDDKPoVUvm+kuxGLnW4W+TFXupTg2OxK6dQD/5t+INCu4nveH9sUHF+wX2K+9QfJ+hQ2MKZzQKRTdiSw7UACUcNjAQfyk5N7aZueXJE8tMeJLegVYa6Y3IQ8O6VYkg/1S719+CwlE47Z5eo5fw4mWyr0d05Et2Lt/ETomhpoMcjpajJAMwI3Nv0Y9heuwTC0uy2cv4X7ZbPOBvtTcDH936FImQRUHus40WEKA2eweEFAQliHxVhFEh6Zd07mxdOsR2LAI3gBEzdzmELd3DVtnxTsSe9hbHy7cvzFDHpKdXtJmyiAJsR38ij915/7Mv5g4np8Espcu6V0pdB/H5QM+pqrrsQ2cEpN6VExT8vRaswrZbqp6Juax4UGyG8KPS7mzV13qRsOIKn3AMA9Z2tjXXCTckqRqZhDIBPply1S+QLE8Ik4Yi3SkfkIjElEuWr4fLSPslkBSfPWeNFNOo84+rFrJnMoG2m4Rp5sLxIL811VYTicJSS6x0Kz1Uqf3LFFGi+Un8wodUahNqpb[r][n]"
107818:07:10,421 DEBUG wire:72 -  >> "[r][n]"
107818:07:10,421 DEBUG headers:272 - >> GET /solr/test3/select?q=*%3A*&_stateVer_=test3%3A46&wt=javabin&version=2 HTTP/1.1
107818:07:10,421 DEBUG headers:275 - >> User-Agent: Solr[org.apache.solr.client.solrj.impl.HttpSolrClient] 1.0
107818:07:10,421 DEBUG headers:275 - >> Host: 10-0-0-64.company.com:18987
107818:07:10,421 DEBUG headers:275 - >> Connection: Keep-Alive
107818:07:10,421 DEBUG headers:275 - >> Authorization: Negotiate YIIFCQYGKwYBBQUCoIIE/TCCBPmgDTALBgkqhkiG9xIBAgKhBAMCAfaiggTgBIIE3GCCBNgGCSqGSIb3EgECAgEAboIExzCCBMOgAwIBBaEDAgEOogcDBQAgAAAAo4IBU2GCAU8wggFLoAMCAQWhDxsNU0VDVVJPTklYLkNPTaIqMCigAwIBAKEhMB8bBEhUVFAbFzEwLTAtMC02NC5zZWN1cm9uaXguY29to4IBBTCCAQGgAwIBEqEDAgEBooH0BIHxKKe1e1rmBivsMS7EM7QlfH/1P9AqnSEuZF/A9o2FBwt6X010f7braFpqBT7mXTU28PiUYecdtxva39av/UUQbSupla64CvR75PatuwiQYKtvnEBHINhX4/lv25Mm6SR7YiFM9o4jbxYXL9k/1bS6UitipNxA7xAX1cLLE270/j/3M4BpL3TgohZR9RVpFrL5rmGTVnrcP6nTZ0YSsaQ4js8fbldYQCxBAyTeiIwb3HGmrl68P/crWaQqv/9dkDS0yuvW26nQoTIwUuvlsTmYAB/Pgopi1hobiTaiMN7z7BIjBXjNBYUPTc6Ab9qxmtaQfqSCA1UwggNRoAMCARKiggNIBIIDROkC2Wypm9nMm60R0rZG+MkcbLLIjyqfSoliOrfCov+0Qko2nxpwOcm1ckdtNgKRnYzzCi9gWTSd9ZIisMPynY/YgCfc4a10QqKER+NwdJLxEXiNfqBKQ81xsp6llZ/I+WMpuZtmH5o8RAtAmamt4G0O31jtBwrSGFMzsxVYGi1kZeN0RqB9TvHScbFXe/TSJWoXc3qL/RzDz519MXQ/MTwYE9klvf15zXgakuEMPJbx0jJ7sVvwRTuNJ+mpv37qwX8JAKMls0045MJlc0zl/39OSh9edjnfi+IIM0K5FJ/wGgm3Vlt9t/rlCuYLlybfB5a3wZ78d/KOhmZwtBQ5arN/vXqE/UQz9rzRsbByZ7kDksphd98TgY3uPaq+SA1MBG5NrawLXxNpoxZZbgob6n6FLzesXQ2dnVMYuGLKoB/VZigAOqMR2wgLNCcNzI+dTvI7gTWf/ao/XfT5EFCd17kRH1XTvhaI6A1zULaGMNxzcoZJDlHZ0AHAnsUj4X0u4xXtVX3LaFKGkXtQNGQEsCtnQbygphl/rrgH06Mv3duhZiNqpb9TVpfbyJ6NoWzX5O4ud+meyAW4rDDKPoVUvm+kuxGLnW4W+TFXupTg2OxK6dQD/5t+INCu4nveH9sUHF+wX2K+9QfJ+hQ2MKZzQKRTdiSw7UACUcNjAQfyk5N7aZueXJE8tMeJLegVYa6Y3IQ8O6VYkg/1S719+CwlE47Z5eo5fw4mWyr0d05Et2Lt/ETomhpoMcjpajJAMwI3Nv0Y9heuwTC0uy2cv4X7ZbPOBvtTcDH936FImQRUHus40WEKA2eweEFAQliHxVhFEh6Zd07mxdOsR2LAI3gBEzdzmELd3DVtnxTsSe9hbHy7cvzFDHpKdXtJmyiAJsR38ij915/7Mv5g4np8Espcu6V0pdB/H5QM+pqrrsQ2cEpN6VExT8vRaswrZbqp6Juax4UGyG8KPS7mzV13qRsOIKn3AMA9Z2tjXXCTckqRqZhDIBPply1S+QLE8Ik4Yi3SkfkIjElEuWr4fLSPslkBSfPWeNFNOo84+rFrJnMoG2m4Rp5sLxIL811VYTicJSS6x0Kz1Uqf3LFFGi+Un8wodUahNqpb
107818:07:10,428 DEBUG wire:72 -  << "HTTP/1.1 200 OK[r][n]"
108518:07:10,428 DEBUG wire:72 -  << "WWW-Authenticate: Negotiate oYH1MIHyoAMKAQChCwYJKoZIhvcSAQICom4EbGBqBgkqhkiG9xIBAgICAG9bMFmgAwIBBaEDAgEPok0wS6ADAgESokQEQlOz50MemlJ23yLycPU2zNSptyEvKzbtbWCzmStU9JF5m1stDqbNn/Z4z0X8Sh8hLZTxBN8Lw0it74YcSnFgBqC2CKNuBGxgagYJKoZIhvcSAQICAgBvWzBZoAMCAQWhAwIBD6JNMEugAwIBEqJEBEJTs+dDHppSdt8i8nD1NszUqbchLys27W1gs5krVPSReZtbLQ6mzZ/2eM9F/EofIS2U8QTfC8NIre+GHEpxYAagtgg=[r][n]"
108518:07:10,428 DEBUG wire:72 -  << "Set-Cookie: hadoop.auth="u=company&p=company@company.COM&t=kerberos-dt&e=1525853230424&s=c7ujCu4e2i31H4l+8cDxxPnOf08="; Path=/; Domain=10-0-0-64.company.com:18987; Expires=Wed, 09-May-2018 08:07:10 GMT; Secure; HttpOnly[r][n]"
108518:07:10,429 DEBUG wire:72 -  << "Content-Type: application/octet-stream[r][n]"
108618:07:10,429 DEBUG wire:72 -  << "Content-Length: 243[r][n]"
108618:07:10,429 DEBUG wire:72 -  << "[r][n]"
108618:07:10,429 DEBUG DefaultClientConnection:253 - Receiving response: HTTP/1.1 200 OK
108618:07:10,429 DEBUG headers:256 - << HTTP/1.1 200 OK
108618:07:10,429 DEBUG headers:259 - << WWW-Authenticate: Negotiate oYH1MIHyoAMKAQChCwYJKoZIhvcSAQICom4EbGBqBgkqhkiG9xIBAgICAG9bMFmgAwIBBaEDAgEPok0wS6ADAgESokQEQlOz50MemlJ23yLycPU2zNSptyEvKzbtbWCzmStU9JF5m1stDqbNn/Z4z0X8Sh8hLZTxBN8Lw0it74YcSnFgBqC2CKNuBGxgagYJKoZIhvcSAQICAgBvWzBZoAMCAQWhAwIBD6JNMEugAwIBEqJEBEJTs+dDHppSdt8i8nD1NszUqbchLys27W1gs5krVPSReZtbLQ6mzZ/2eM9F/EofIS2U8QTfC8NIre+GHEpxYAagtgg=
108618:07:10,429 DEBUG headers:259 - << Set-Cookie: hadoop.auth="u=company&p=company@company.COM&t=kerberos-dt&e=1525853230424&s=c7ujCu4e2i31H4l+8cDxxPnOf08="; Path=/; Domain=10-0-0-64.company.com:18987; Expires=Wed, 09-May-2018 08:07:10 GMT; Secure; HttpOnly
108618:07:10,429 DEBUG headers:259 - << Content-Type: application/octet-stream
108618:07:10,430 DEBUG headers:259 - << Content-Length: 243
108718:07:10,430 DEBUG ResponseProcessCookies:118 - Cookie accepted [hadoop.auth=""u=company&p=company@company.COM&t=kerberos-dt&e=1525853230424&s=c7ujCu4e2i31H4l+8cDxxPnOf08="", version:0, domain:10-0-0-64.company.com:18987, path:/, expiry:Wed May 09 04:07:10 EDT 2018]
108718:07:10,431 DEBUG SystemDefaultHttpClient:510 - Connection can be kept alive indefinitely
108818:07:10,431 DEBUG SystemDefaultHttpClient:86 - Authentication succeeded
108818:07:10,446 DEBUG wire:86 -  << "[0x2][0xa2][0xe0].responseHeader[0xa4][0xe0]+zkConnected[0x1][0xe0]&status[0x6][0x0][0x0][0x0][0x0][0xe0]%QTime[0x6][0x0][0x0][0x0][0x0][0xe0]&params[0xa4][0xe0]!q#*:*[0xe0]*_stateVer_(test3:46[0xe0]"wt'javabin[0xe0]'version!2[0xe0](response[0xc][0x83]b`[0x0][0x82][0xb][0xa5][0xe0]-useruniquekey#1~1[0xe0]$u_ida[0xe0])_version_[0x7][0x16]/Kk[0xab] [0x0][0x0][0xe0](tenantida[0xe0]*tenantname)company[0xb][0xa4][0xeb]!1[0xed][0x7][0x16]4[0x18][0x9c]C`[0x0][0x0][0xee]a[0xef])company"
110318:07:10,448 DEBUG PoolingClientConnectionManager:262 - Connection [id: 0][route: {s}->https://10-0-0-64.company.com:18987][state: class org.apache.solr.client.solrj.impl.HttpSolrClient] can be kept alive indefinitely

这可能是由 Kerberos 客户端和服务器之间的加密类型不匹配引起的

尝试更改/etc/krb5.conf - 在 Solr 服务器和客户端计算机上 - 添加 [libdefaults] 部分

[库默认]

default_tkt_enctypes = 弧四-HMAC-MD5

相关内容

最新更新

热门标签：