Kubernetes dashboard nodeport 不起作用



我正在尝试使用 NodePort 运行 kubernetes 仪表板。但这似乎不起作用,我正在按照以下步骤操作:

  1. 通过 kubeadm 版本 1.15.3 安装 kubernetes
  2. 将法兰绒安装为吊舱网络
  3. 安装端口类型为 NodePort 的 kubernetes 仪表板

当我做kubctl logs ... kubernetes-dashboard时,我得到以下结果

2019/09/03 01:22:31 Starting overwatch
2019/09/03 01:22:31 Using namespace: kubernetes-dashboard
2019/09/03 01:22:31 Using in-cluster config to connect to apiserver
2019/09/03 01:22:31 Using secret token for csrf signing
2019/09/03 01:22:31 Initializing csrf token from kubernetes-dashboard-csrf secret
2019/09/03 01:22:31 Empty token. Generating and storing in a secret kubernetes-dashboard-csrf
2019/09/03 01:22:31 Successful initial request to the apiserver, version: v1.15.3
2019/09/03 01:22:31 Generating JWE encryption key
2019/09/03 01:22:31 New synchronizer has been registered: kubernetes-dashboard-key-holder-kubernetes-dashboard. Starting
2019/09/03 01:22:31 Starting secret synchronizer for kubernetes-dashboard-key-holder in namespace kubernetes-dashboard
2019/09/03 01:22:32 Initializing JWE encryption key from synchronized object
2019/09/03 01:22:32 Creating in-cluster Sidecar client
2019/09/03 01:22:32 Auto-generating certificates
2019/09/03 01:22:32 Successfully created certificates
2019/09/03 01:22:32 Serving securely on HTTPS port: 8443
2019/09/03 01:22:32 Successful request to sidecar

到目前为止,对我来说似乎很好。 现在要获取节点端口,我kubectl get svc --namespace kubernetes-dashboard

现在我得到

NAME                        TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)         AGE
dashboard-metrics-scraper   ClusterIP   10.108.213.154   <none>        8000/TCP        11m
kubernetes-dashboard        NodePort    10.109.111.248   <none>        443:30206/TCP   11m

现在,当我访问https://master-ip:30206时,我只是超时,日志保持不变。

服务亚姆:

kind: Service
apiVersion: v1
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kubernetes-dashboard
spec:
type: NodePort
ports:
- port: 443
targetPort: 8443
selector:
k8s-app: kubernetes-dashboard

部署 yaml:

kind: Deployment
apiVersion: apps/v1
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kubernetes-dashboard
spec:
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
k8s-app: kubernetes-dashboard
template:
metadata:
labels:
k8s-app: kubernetes-dashboard
spec:
containers:
- name: kubernetes-dashboard
image: kubernetesui/dashboard:v2.0.0-beta4
imagePullPolicy: Always
ports:
- containerPort: 8443
protocol: TCP
args:
- --auto-generate-certificates
- --namespace=kubernetes-dashboard
# Uncomment the following line to manually specify Kubernetes API server Host
# If not specified, Dashboard will attempt to auto discover the API server and connect
# to it. Uncomment only if the default does not work.
# - --apiserver-host=http://my-address:port
volumeMounts:
- name: kubernetes-dashboard-certs
mountPath: /certs
# Create on-disk volume to store exec logs
- mountPath: /tmp
name: tmp-volume
livenessProbe:
httpGet:
scheme: HTTPS
path: /
port: 8443
initialDelaySeconds: 30
timeoutSeconds: 30
volumes:
- name: kubernetes-dashboard-certs
secret:
secretName: kubernetes-dashboard-certs
- name: tmp-volume
emptyDir: {}
serviceAccountName: kubernetes-dashboard
# Comment the following tolerations if Dashboard must not be deployed on master
tolerations:
- key: node-role.kubernetes.io/master
effect: NoSchedule

应用yaml时的输出:

:namespace/kubernetes-dashboard created
serviceaccount/kubernetes-dashboard created
service/kubernetes-dashboard created
secret/kubernetes-dashboard-certs created
secret/kubernetes-dashboard-csrf created
secret/kubernetes-dashboard-key-holder created
configmap/kubernetes-dashboard-settings created
role.rbac.authorization.k8s.io/kubernetes-dashboard created
clusterrole.rbac.authorization.k8s.io/kubernetes-dashboard created
rolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created
clusterrolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created
deployment.apps/kubernetes-dashboard created
service/dashboard-metrics-scraper created
deployment.apps/dashboard-metrics-scraper created

我终于能够解决这个问题,并采取了以下步骤:

我的iptables似乎搞砸了,当我在主服务器上执行以下命令时,我发现了这一点:

curl {master-ip/e.g 192.168.1.11}:{Nodeport of kubernetes dashboard}

这个命令正在工作,我正在恢复仪表板,同样的 curl 让我在我的客户端计算机上超时。

所以我通过在主节点上执行以下命令来解决此问题:

sudo iptables -A FORWARD -j ACCEPT

现在出现了另一个问题,当您访问chrome时会给出无效的证书错误

要解决此问题,需要执行以下操作:

创建一个名为 san.cnf 的文件,其中包含以下内容:

[ req ]
default_bits        = 2048
distinguished_name  = req_distinguished_name
req_extensions      = req_ext
prompt              = no
[ req_distinguished_name ]
countryName             = NL
stateOrProvinceName     = Noord Brabant
localityName            = Rosmalen
organizationName        = ARRServices
commonName              = k8s.dashboard.prod
[ req_ext ]
subjectAltName = @alt_names
[alt_names]
DNS.1   = {optional only do this if you reverse proxy}
IP.1    = {IP of the machine that is running the dashboard usually the master node}
# Create certificate by executing the following commands
sudo mkdir /certs    
sudo chmod 777 -R /certs
openssl req -out /certs/dashboard.csr -newkey rsa:2048 -nodes -keyout /certs/dashboard.key -config san.cnf
openssl x509 -req -sha256 -days 3650 -in /certs/dashboard.csr -signkey /certs/dashboard.key -out /certs/dashboard.crt -extensions req_ext -extfile san.cnf
sudo chmod 777 -R /certs

现在你的 kubernetes-dashboard.yml 文件应该看起来像这样:

apiVersion: v1
kind: Namespace
metadata:
name: kubernetes-dashboard
---
apiVersion: v1
kind: ServiceAccount
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kubernetes-dashboard
---
kind: Service
apiVersion: v1
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kubernetes-dashboard
spec:
type: NodePort
ports:
- port: 443
targetPort: 8443
nodePort: 31000
selector:
k8s-app: kubernetes-dashboard
---
apiVersion: v1
kind: Secret
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard-certs
namespace: kubernetes-dashboard
type: Opaque
---
apiVersion: v1
kind: Secret
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard-csrf
namespace: kubernetes-dashboard
type: Opaque
data:
csrf: ""
---
apiVersion: v1
kind: Secret
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard-key-holder
namespace: kubernetes-dashboard
type: Opaque
---
kind: ConfigMap
apiVersion: v1
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard-settings
namespace: kubernetes-dashboard
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kubernetes-dashboard
rules:
# Allow Dashboard to get, update and delete Dashboard exclusive secrets.
- apiGroups: [""]
resources: ["secrets"]
resourceNames: ["kubernetes-dashboard-key-holder", "kubernetes-dashboard-certs", "kubernetes-dashboard-csrf"]
verbs: ["get", "update", "delete"]
# Allow Dashboard to get and update 'kubernetes-dashboard-settings' config map.
- apiGroups: [""]
resources: ["configmaps"]
resourceNames: ["kubernetes-dashboard-settings"]
verbs: ["get", "update"]
# Allow Dashboard to get metrics.
- apiGroups: [""]
resources: ["services"]
resourceNames: ["heapster", "dashboard-metrics-scraper"]
verbs: ["proxy"]
- apiGroups: [""]
resources: ["services/proxy"]
resourceNames: ["heapster", "http:heapster:", "https:heapster:", "dashboard-metrics-scraper", "http:dashboard-metrics-scraper"]
verbs: ["get"]
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
rules:
# Allow Metrics Scraper to get metrics from the Metrics server
- apiGroups: ["metrics.k8s.io"]
resources: ["pods", "nodes"]
verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kubernetes-dashboard
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: kubernetes-dashboard
subjects:
- kind: ServiceAccount
name: kubernetes-dashboard
namespace: kubernetes-dashboard
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: kubernetes-dashboard
namespace: kubernetes-dashboard
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: kubernetes-dashboard
subjects:
- kind: ServiceAccount
name: kubernetes-dashboard
namespace: kubernetes-dashboard
---
kind: Deployment
apiVersion: apps/v1
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kubernetes-dashboard
spec:
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
k8s-app: kubernetes-dashboard
template:
metadata:
labels:
k8s-app: kubernetes-dashboard
spec:
containers:
- name: kubernetes-dashboard
image: kubernetesui/dashboard:v2.0.0-beta4
imagePullPolicy: Always
ports:
- containerPort: 8443
protocol: TCP
args:
- --auto-generate-certificates=false
- --namespace=kubernetes-dashboard
- --tls-cert-file=dashboard.crt
- --tls-key-file=dashboard.key
# Uncomment the following line to manually specify Kubernetes API server Host
# If not specified, Dashboard will attempt to auto discover the API server and connect
# to it. Uncomment only if the default does not work.
# - --apiserver-host=http://my-address:port
volumeMounts:
- name: kubernetes-dashboard-certs
mountPath: /certs
# Create on-disk volume to store exec logs
- mountPath: /tmp
name: tmp-volume
livenessProbe:
httpGet:
scheme: HTTPS
path: /
port: 8443
initialDelaySeconds: 30
timeoutSeconds: 30
volumes:
- name: kubernetes-dashboard-certs
hostPath:
path: /certs
#secret:
#  secretName: kubernetes-dashboard-certs
- name: tmp-volume
emptyDir: {}
serviceAccountName: kubernetes-dashboard
# Comment the following tolerations if Dashboard must not be deployed on master
tolerations:
- key: node-role.kubernetes.io/master
effect: NoSchedule
---
kind: Service
apiVersion: v1
metadata:
labels:
k8s-app: dashboard-metrics-scraper
name: dashboard-metrics-scraper
namespace: kubernetes-dashboard
spec:
ports:
- port: 8000
targetPort: 8000
selector:
k8s-app: dashboard-metrics-scraper
---
kind: Deployment
apiVersion: apps/v1
metadata:
labels:
k8s-app: dashboard-metrics-scraper
name: dashboard-metrics-scraper
namespace: kubernetes-dashboard
spec:
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
k8s-app: dashboard-metrics-scraper
template:
metadata:
labels:
k8s-app: dashboard-metrics-scraper
spec:
containers:
- name: dashboard-metrics-scraper
image: kubernetesui/metrics-scraper:v1.0.1
ports:
- containerPort: 8000
protocol: TCP
livenessProbe:
httpGet:
scheme: HTTP
path: /
port: 8000
initialDelaySeconds: 30
timeoutSeconds: 30
volumeMounts:
- mountPath: /tmp
name: tmp-volume
serviceAccountName: kubernetes-dashboard
# Comment the following tolerations if Dashboard must not be deployed on master
tolerations:
- key: node-role.kubernetes.io/master
effect: NoSchedule
volumes:
- name: tmp-volume
emptyDir: {}

我所做的更改如下: Kubernetes-dashboard的服务将具有:

NodePort 而不是 ClusterIP

  • 静态节点端口 31000 而不是动态端口
  • Kubernetes-dashboard部署的参数如下:

    - --auto-generate-certificates=false
    - --namespace=kubernetes-dashboard
    - --tls-cert-file=dashboard.crt
    - --tls-key-file=dashboard.key
    
  • 从卷中替换了机密和机密名称 kubernetes-dashboard-certs with hostPath: path:/certs # 指的是 我们生成证书的路径

    - name: kubernetes-dashboard-certs
    hostPath:
    path: /certs
    

现在执行以下命令: 'Kubectl Apply -f kubernetes-dashboard.yml'

现在您将能够通过 {ip}:{nodePort} 访问它

  • IP 通常是主节点 ip
  • 可以通过执行以下操作来检索节点端口:

    kubectl get svc --namespace kubernetes-dashboard

大多数浏览器会给出一个看起来很丑的错误,要解决此问题,请执行以下操作(Mac OSX,使用chrome(:

  • 打开chrome并转到仪表板的IP和端口,例如 "192.168.1.11:31000">
  • 您将看到证书不受信任,因此请下载 证书:转到开发人员工具 => 安全性 =>查看证书, 并将证书图标/徽标拖到桌面
  • 打开钥匙串 => 转到钥匙串"系统" => 转到类别 "证书">
  • 将证书拖到证书列表中
  • 双击证书将其打开,单击"信任" - 选择 在"使用
  • 此证书"的"始终信任"选项

现在您应该能够访问仪表板,下一步是创建管理员用户,这可以按以下步骤完成:

创建一个名为 serviceaccount.yml 的文件,其中包含以下内容

apiVersion: v1
kind: ServiceAccount
metadata:
name: admin-user
namespace: kubernetes-dashboard

创建一个名为 rbac.yml 的文件,其中包含以下内容:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: admin-user
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: admin-user
namespace: kubernetes-dashboard

执行以下命令

kubectl apply -f serviceaccount.yml
kubectl apply -f rbac.yml

通过执行此命令获取令牌

kubectl -n kubernetes-dashboard describe secret $(kubectl -n kubernetes-dashboard get secret | grep admin-user | awk '{print $1}')

这适用于 kubernetes 1.15.3、docker 18.09、kubernetes-dashboard 2.0.0beta4

测试:树莓派 4 作为主节点,但也应该与其他设备配合使用。

您是否为 kubernetes-dashboard serviceaccount 创建了 ClusterRole、ClusterRoleBinding 资源? 如果没有,您可以使用下面的定义文件来应用更改吗?

kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/master/aio/deploy/recommended/05_dashboard-rbac.yaml

最新更新