小贝子编程

成功oauth2后无法加载kubernetes仪表板



我已经用kubeadm部署了我的kubernetes集群v1.23.1，并用密钥斗篷身份提供程序对其进行了配置以进行身份验证。

密钥斗篷IDP 的API服务器配置

...
- --oidc-issuer-url=https://kubemaster:8443/auth/realms/local
- --oidc-client-id=gatekeeper
- --oidc-username-claim=name
- --oidc-groups-claim=groups
- --oidc-ca-file=/etc/kubernetes/ssl/kubemaster.crt
...

我正在另一个pod中运行oauth2代理，该代理使用keycloft idp进行身份验证并提供令牌(id_token(。这是我的oauth2代理部署和公开它的服务

---
kind: Deployment
apiVersion: apps/v1
metadata:
labels:
k8s-app: oauth2-proxy
name: oauth2-proxy
namespace: kubernetes-dashboard
spec:
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
k8s-app: oauth2-proxy
template:
metadata:
labels:
k8s-app: oauth2-proxy
spec:
containers:
- name: oauth2-proxy
image: quay.io/oauth2-proxy/oauth2-proxy:latest
imagePullPolicy: Always
ports:
- containerPort: 4180
protocol: TCP
args:
- --provider=oidc
- --email-domain=*
- --http-address=0.0.0.0:4180
env:
- name: OAUTH2_PROXY_OIDC_ISSUER_URL
value: https://192.168.122.54:8443/auth/realms/local
- name: OAUTH2_PROXY_REDIRECT_URL
value: https://kubernetes-dashboard.localdev.me:8081/oauth2/callback
- name: OAUTH2_PROXY_CLIENT_ID
value: gatekeeper
- name: OAUTH2_PROXY_CLIENT_SECRET
value: jZzvJ0wCDDwltV3tAf0SXSbVoKXM1RqV
- name: OAUTH2_PROXY_COOKIE_SECRET
value: kgKUT3IMmESA81VWXvRpYIYwMSo1xndwIogUks6IS00=
- name: OAUTH2_PROXY_UPSTREAM
value: https://kubernetes-dashboard
- name: OAUTH2_PROXY_SSL_INSECURE_SKIP_VERIFY
value: "true"
- name: OAUTH2_PROXY_COOKIE_DOMAIN
value: .localdev.me
- name: OAUTH2_PROXY_INSECURE_OIDC_ALLOW_UNVERIFIED_EMAIL
value: "true"
- name: OAUTH2_PROXY_PASS_AUTHORIZATION_HEADER
value: "true"
- name: OAUTH2_PROXY_SSL_UPSTREAM_INSECURE_SKIP_VERIFY
value: "true"
- name: OAUTH2_PROXY_OIDC_EMAIL_CLAIM
value: email
- name: OAUTH2_PROXY_GROUPS_CLAIM
value: groups
- name: OAUTH2_PROXY_ALLOWED_GROUPS
value: developers
---
kind: Service
apiVersion: v1
metadata:
labels:
k8s-app: oauth2-proxy
name: oauth2-proxy
namespace: kubernetes-dashboard
spec:
#type: NodePort
ports:
- port: 4180
targetPort: 4180
name: http
selector:
k8s-app: oauth2-proxy

我在前面部署了nginxingress，将请求路由到oauth2代理，如下所示。

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: oauth-proxy
namespace: kubernetes-dashboard
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
ingressClassName: nginx
rules:
- host: kubernetes-dashboard.localdev.me
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: oauth2-proxy
port:
number: 4180
tls:
- hosts:
- kubernetes-dashboard.localdev.me
secretName: kubernetes-dashboard-ingress-tls

我使用端口转发从本地主机到端口443上的入口资源。

kubectl port-forward --namespace=ingress-nginx service/ingress-nginx-controller 8081:443

我目前面临的问题是，我能够启动网址-https://kubernetes-dashboard.localdev.me:8081/它路由到oauth2代理页面，从那里我可以启动我可以验证的密钥斗篷领域登录页面。但是，一旦密钥斗篷身份验证成功完成，我就无法加载带有id_token的上游kubernetes仪表板服务。相反，我得到了一个找不到的404https://kubernetes-dashboard.localdev.me:8081/

oauth2代理日志详细信息

[2022/01/04 19:06:24] [oauthproxy.go:866] No valid authentication in request. Initiating login.
192.168.1.169:33826 - 48490760d6e0fc519e13d3158878112a - - [2022/01/04 19:06:24] kubernetes-dashboard.localdev.me:8081 GET - "/" HTTP/1.1 "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:95.0) Gecko/20100101 Firefox/95.0" 403 8033 0.000
[2022/01/04 19:06:25] [oauthproxy.go:866] No valid authentication in request. Initiating login.
192.168.1.169:33826 - a87fbdc228026d99e6fc9f29d9ffc6df - - [2022/01/04 19:06:25] kubernetes-dashboard.localdev.me:8081 GET - "/favicon.ico" HTTP/1.1 "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:95.0) Gecko/20100101 Firefox/95.0" 403 8044 0.000
192.168.1.169:33826 - 3c052e49f9f77e89c561f3d0bef47fed - - [2022/01/04 19:06:43] kubernetes-dashboard.localdev.me:8081 GET - "/oauth2/start?rd=%2F" HTTP/1.1 "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:95.0) Gecko/20100101 Firefox/95.0" 302 356 0.000
192.168.1.169:33826 - a16b9b1b22ec03e58b9b930caa325f5b - alice@stack.com [2022/01/04 19:06:55] [AuthSuccess] Authenticated via OAuth2: Session{email:alice@stack.com user:52ed0190-4d9c-4749-90f4-37e5896bdc42 PreferredUsername:alice token:true id_token:true created:2022-01-04 19:06:55.77587057 +0000 UTC m=+290.895007232 expires:2022-01-04 19:11:55.775499536 +0000 UTC m=+590.894636200 refresh_token:true groups:[developers]}
192.168.1.169:33826 - a16b9b1b22ec03e58b9b930caa325f5b - - [2022/01/04 19:06:55] kubernetes-dashboard.localdev.me:8081 GET - "/oauth2/callback?state=HBDoNuX3mudlwQrjdEkxYP9yjgHn5_mqOXk8T-G21dg%3A%2F&session_state=e928739a-c94d-49ba-80a9-a1f83a936bf1&code=9a592559-236c-4896-86b4-ba8f657821e6.e928739a-c94d-49ba-80a9-a1f83a936bf1.b7b62a18-269b-480a-b5bd-ad3c16f94394" HTTP/1.1 "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:95.0) Gecko/20100101 Firefox/95.0" 302 24 0.030
192.168.1.169:33826 - 9288833c2f8e82f2ad94c6260bcf68c7 - alice@stack.com [2022/01/04 19:06:55] kubernetes-dashboard.localdev.me:8081 GET - "/" HTTP/1.1 "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:95.0) Gecko/20100101 Firefox/95.0" 404 19 0.000

入口控制器记录

127.0.0.1 - - [04/Jan/2022:19:02:26 +0000] "GET / HTTP/2.0" 404 19 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:95.0) Gecko/20100101 Firefox/95.0" 2842 0.001 [kubernetes-dashboard-oauth2-proxy-4180] [] 192.168.1.177:4180 19 0.000 404 d19c25394498799f5024567ddd1fafcd
127.0.0.1 - - [04/Jan/2022:19:06:24 +0000] "GET / HTTP/2.0" 403 8033 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:95.0) Gecko/20100101 Firefox/95.0" 297 0.001 [kubernetes-dashboard-oauth2-proxy-4180] [] 192.168.1.177:4180 8060 0.000 403 48490760d6e0fc519e13d3158878112a
127.0.0.1 - - [04/Jan/2022:19:06:25 +0000] "GET /favicon.ico HTTP/2.0" 403 8044 "https://kubernetes-dashboard.localdev.me:8081/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:95.0) Gecko/20100101 Firefox/95.0" 102 0.003 [kubernetes-dashboard-oauth2-proxy-4180] [] 192.168.1.177:4180 8071 0.004 403 a87fbdc228026d99e6fc9f29d9ffc6df
127.0.0.1 - - [04/Jan/2022:19:06:43 +0000] "GET /oauth2/start?rd=%2F HTTP/2.0" 302 356 "https://kubernetes-dashboard.localdev.me:8081/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:95.0) Gecko/20100101 Firefox/95.0" 32 0.001 [kubernetes-dashboard-oauth2-proxy-4180] [] 192.168.1.177:4180 356 0.000 302 3c052e49f9f77e89c561f3d0bef47fed
127.0.0.1 - - [04/Jan/2022:19:06:55 +0000] "GET /oauth2/callback?state=HBDoNuX3mudlwQrjdEkxYP9yjgHn5_mqOXk8T-G21dg%3A%2F&session_state=e928739a-c94d-49ba-80a9-a1f83a936bf1&code=9a592559-236c-4896-86b4-ba8f657821e6.e928739a-c94d-49ba-80a9-a1f83a936bf1.b7b62a18-269b-480a-b5bd-ad3c16f94394 HTTP/2.0" 302 24 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:95.0) Gecko/20100101 Firefox/95.0" 351 0.031 [kubernetes-dashboard-oauth2-proxy-4180] [] 192.168.1.177:4180 24 0.028 302 a16b9b1b22ec03e58b9b930caa325f5b
127.0.0.1 - - [04/Jan/2022:19:06:55 +0000] "GET / HTTP/2.0" 404 19 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:95.0) Gecko/20100101 Firefox/95.0" 2827 0.001 [kubernetes-dashboard-oauth2-proxy-4180] [] 192.168.1.177:4180 19 0.004 404 9288833c2f8e82f2ad94c6260bcf68c7

请建议我如何将带有id_token的已验证会话路由到kubernetes仪表板服务以加载仪表板。

我终于能够通过更新我的yaml定义文件来解决我的问题。

假设你在Ubuntu 20.04上安装了一个带有kubeadm的kubernetes集群v1.23.1，并使用法兰绒网络设置了网络——pod网络cidr=10.244.0.0/16。此外，您还拥有keycaplo-oidc服务设置(image-quay.io/keycaplo/keycapro:116.1.0(。以下是帮助我解决此问题的更新的yml定义文件。

应用入口控制器-

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.1.0/deploy/static/provider/cloud/deploy.yaml

kubernetes仪表板应用-

kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.4.0/aio/deploy/recommended.yaml

oauth2-proxy.yml

apiVersion: apps/v1
kind: Deployment
metadata:
labels:
k8s-app: oauth2-proxy
name: oauth2-proxy
namespace: kubernetes-dashboard
spec:
replicas: 1
selector:
matchLabels:
k8s-app: oauth2-proxy
template:
metadata:
labels:
k8s-app: oauth2-proxy
spec:
containers:
- args:
- --provider=oidc
- --email-domain=*
- --http-address=0.0.0.0:4180
env:
- name: OAUTH2_PROXY_OIDC_ISSUER_URL
value: https://kubemaster:8443/auth/realms/local
- name: OAUTH2_PROXY_REDIRECT_URL
value: https://<FQDN>/oauth2/callback
- name: OAUTH2_PROXY_CLIENT_ID
value: gatekeeper
- name: OAUTH2_PROXY_CLIENT_SECRET
value: jZzvJ0wCDDwltV3tAf0SXSbVoKXM1RqV
- name: OAUTH2_PROXY_COOKIE_SECRET
value: kgKUT3IMmESA81VWXvRpYIYwMSo1xndwIogUks6IS00=
- name: OAUTH2_PROXY_UPSTREAM
value: https://kubernetes-dashboard
- name: OAUTH2_PROXY_SSL_INSECURE_SKIP_VERIFY
value: "true"
- name: OAUTH2_PROXY_INSECURE_OIDC_ALLOW_UNVERIFIED_EMAIL
value: "true"
- name: OAUTH2_PROXY_PASS_AUTHORIZATION_HEADER
value: "true"
- name: OAUTH2_PROXY_SSL_UPSTREAM_INSECURE_SKIP_VERIFY
value: "true"
- name: OAUTH2_PROXY_OIDC_EMAIL_CLAIM
value: email
- name: OAUTH2_PROXY_GROUPS_CLAIM
value: groups
- name: OAUTH2_PROXY_ALLOWED_GROUPS
value: developers
- name: OAUTH2_PROXY_SKIP_PROVIDER_BUTTON
value: "true"
- name: OAUTH2_PROXY_SET_AUTHORIZATION_HEADER
value: "true"
image: quay.io/oauth2-proxy/oauth2-proxy:latest
imagePullPolicy: Always
name: oauth2-proxy
ports:
- containerPort: 4180
protocol: TCP

dashboard-ingress.yml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/auth-url: "https://$host/oauth2/auth"
nginx.ingress.kubernetes.io/auth-signin: "https://$host/oauth2/start?rd=$escaped_request_uri"
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
nginx.ingress.kubernetes.io/auth-response-headers: "authorization"
name: external-auth-oauth2
namespace: kubernetes-dashboard
spec:
ingressClassName: nginx
rules:
- host: __INGRESS_HOST__
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: kubernetes-dashboard
port:
number: 443
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: oauth2-proxy
namespace: kubernetes-dashboard
spec:
ingressClassName: nginx
rules:
- host: __INGRESS_HOST__
http:
paths:
- path: /oauth2
pathType: Prefix
backend:
service:
name: oauth2-proxy
port:
number: 4180
tls:
- hosts:
- __INGRESS_HOST__
secretName: __INGRESS_SECRET__

密钥斗篷oidc提供程序的问题是无法设置Authroization标头，为此我们需要在oauth2代理定义中手动添加设置的授权标头，并且需要通过设置annotation-nginx.ingress.kubernetes.io/auth-response-headers将此标头添加到从入口到上游kubernetes面板的响应中："；"授权"；

还添加了一个使用github oauth2进行kubernetes仪表板身份验证的参考，我遵循该参考来解决此问题。

https://kubernetes.github.io/ingress-nginx/examples/auth/oauth-external-auth/#example-oauth2-proxy-kubernetes-dashboard

相关内容

最新更新


热门标签：

javascript python java c# php android html jquery c++ css ios sql mysql arrays asp.net json python-3.x ruby-on-rails .net sql-server django objective-c excel regex ruby linux ajax iphone xml vba spring asp.net-mvc database wordpress string postgresql wpf windows xcode bash git oracle list vb.net multithreading eclipse algorithm macos powershell visual-studio image forms numpy scala function api selenium