kubectl 代理无法与 API 服务器通信

Question

我在全新安装时遇到kubectl proxy问题。

当我浏览到 http://localhost:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/时，我收到 503 响应。似乎代理无法访问 kubernetes API，即使其他命令可以访问。

Kubernetes 运行在 DC/OS 中，包为 1.3.1-1.10.8。kubectl 和 Kubernetes 都是 1.10.8 版本。DC/OS 中配置了一个负载均衡器来公开 API。

LB 定义来自 kubernetes on dcos 帮助页面。我在标签中添加了"HAPROXY_0_VHOST": "k8s-proxy.dcos.<domain>.com"。

$ kubectl cluster-info
Kubernetes master is running at https://k8s-proxy.dcos.<domain>.com
KubeDNS is running at https://k8s-proxy.dcos.<domain>.com/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

我在详细输出模式下运行kubectl proxy，看看它试图进行什么调用。它得到了503响应。

$ kubectl proxy --insecure-skip-tls-verify=true --alsologtostderr=true -v=99
I0327 12:26:45.461259   19980 loader.go:357] Config loaded from file U:/.kube/config
Starting to serve on 127.0.0.1:8001
I0327 12:26:56.200819   19980 proxy_server.go:98] /api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/ matched ^.*
I0327 12:26:56.200819   19980 proxy_server.go:98] localhost matched ^localhost$
I0327 12:26:56.200819   19980 proxy_server.go:138] Filter accepting GET /api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/ localhost
I0327 12:26:56.200819   19980 upgradeaware.go:237] Request was not an upgrade
I0327 12:26:56.200819   19980 round_trippers.go:387] curl -k -v -XGET  -H "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8" -H "Cache-Control: max-age=0" -H "User-Agent: Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36" -H "Authorization: Bearer <my_token>" -H "X-Forwarded-For: 127.0.0.1" -H "Accept-Language: en-US,en;q=0.9" -H "Dnt: 1" -H "Accept-Encoding: gzip, deflate, br" -H "Upgrade-Insecure-Requests: 1" https://k8s-proxy.dcos.<domain>.com/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/
I0327 12:26:56.313141   19980 round_trippers.go:406] GET https://k8s-proxy.dcos.<domain>.com/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/ 503 Service Unavailable in 112 milliseconds
I0327 12:26:56.313141   19980 round_trippers.go:412] Response Headers:
I0327 12:26:56.313141   19980 round_trippers.go:415]     Cache-Control: no-cache
I0327 12:26:56.313141   19980 round_trippers.go:415]     Content-Type: text/html

在同一个外壳中，我尝试运行代理运行的卷曲。它得到了200而不是503。

$ curl -k -v -XGET  -H "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8" -H "Cache-Control: max-age=0" -H "User-Agent: Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36" -H "Authorization: Bearer <my_token>" -H "X-Forwarded-For: 127.0.0.1" -H "Accept-Language: en-US,en;q=0.9" -H "Dnt: 1" -H "Accept-Encoding: gzip, deflate, br" -H "Upgrade-Insecure-Requests: 1" https://k8s-proxy.dcos.<domain>.com/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/
[...]
< HTTP/1.1 200 OK
< Accept-Ranges: bytes
< Cache-Control: no-store
< Content-Encoding: gzip
< Content-Type: text/html; charset=utf-8
< Date: Wed, 27 Mar 2019 19:30:24 GMT
< Last-Modified: Fri, 24 Aug 2018 05:39:29 GMT
< Content-Length: 529
[...]

我希望能够访问我的集群，但有效的请求返回 503。其他kubectl命令工作正常。这不是特定于仪表板的问题。

Answer 1

最常见的

问题，在部署仪表板时缺少向服务帐户授予权限以管理 kube-system 命名空间中的机密。更多细节在这里

这会导致仪表板 Pod 陷入crashloopbackoff，从而导致仪表板服务缺少终结点。

因此，对其进行故障排除的第一步是检查端点

kubectl get ep -n kube-system kubernetes-dashboard