对 kube-dns pod 进行故障排除:等待初始化超时



我的 kube-dns pod 崩溃了:

kube-dns-6d4fc847dc-6bh59                               1/3     CrashLoopBackOff   5844       7d13h

这些是来自 kubedns 容器的日志

kubectl logs kube-dns-6d4fc847dc-6bh59 --namespace kube-system  -c kubedns
I1021 00:56:37.547936       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:56:38.047923       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
E1021 00:56:38.048390       1 reflector.go:201] k8s.io/dns/pkg/dns/dns.go:192: Failed to list *v1.Service: Get https://100.64.0.1:443/api/v1/services?resourceVersion=0: dial tcp 100.64.0.1:443: i/o timeout
E1021 00:56:38.048405       1 reflector.go:201] k8s.io/dns/pkg/dns/dns.go:189: Failed to list *v1.Endpoints: Get https://100.64.0.1:443/api/v1/endpoints?resourceVersion=0: dial tcp 100.64.0.1:443: i/o timeout
I1021 00:56:38.547922       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:56:39.047917       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:56:39.547923       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:56:40.047935       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:56:40.547928       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:56:41.047931       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:56:41.547929       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:56:42.047924       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:56:42.547931       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:56:43.047909       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:56:43.547921       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:56:44.047923       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:56:44.547932       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:56:45.047935       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:56:45.547927       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:56:46.047928       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:56:46.547929       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:56:47.047925       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:56:47.547933       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:56:48.047914       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:56:48.547930       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:56:49.047927       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:56:49.547929       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:56:50.047925       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:56:50.547928       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:56:51.047933       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:56:51.547928       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:56:52.047920       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:56:52.547930       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:56:53.047913       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:56:53.547931       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:56:54.047925       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:56:54.547930       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:56:55.047928       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:56:55.547938       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:56:56.047935       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:56:56.547937       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:56:57.047940       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:56:57.547941       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:56:58.047915       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:56:58.547934       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:56:59.047932       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:56:59.547937       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:57:00.047942       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:57:00.547934       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:57:01.047925       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:57:01.547929       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:57:02.047915       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:57:02.547924       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:57:03.047906       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:57:03.547927       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:57:04.047934       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:57:04.547927       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:57:05.047933       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:57:05.547925       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:57:06.047928       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:57:06.547926       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:57:07.047926       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
I1021 00:57:07.547930       1 dns.go:219] Waiting for [endpoints services] to be initialized from apiserver...
F1021 00:57:08.047911       1 dns.go:209] Timeout waiting for initialization

请帮忙。

看起来你的kube-dnspod 无法与你的 kube-apiserver 通信。kube-apiserver 上的端口443已被弃用了一段时间。您的配置映射或配置文件使其指向<kube-master:443>

您可以尝试使用--kube-master-url选项在 kube-dns 部署上覆盖它。像这样:

apiVersion: apps/v1
kind: Deployment
metadata:
name: kube-dns
namespace: kube-system
labels:
k8s-app: kube-dns
kubernetes.io/cluster-service: "true"
addonmanager.kubernetes.io/mode: Reconcile
spec:
strategy:
rollingUpdate:
maxSurge: 10%
maxUnavailable: 0
selector:
matchLabels:
k8s-app: kube-dns
template:
metadata:
labels:
k8s-app: kube-dns
annotations:
scheduler.alpha.kubernetes.io/critical-pod: ''
spec:
tolerations:
- key: "CriticalAddonsOnly"
operator: "Exists"
volumes:
- name: kube-dns-config
configMap:
name: kube-dns
optional: true
containers:
- name: kubedns
image: gcr.io/google_containers/k8s-dns-kube-dns-amd64:1.14.7
...
args:
- --kube-master-url=https://<master-ip>:6443
- --domain=cluster.local.
- --dns-port=10053
- --config-dir=/kube-dns-config
- --v=2

您可以从以下输出中获取:

$ kubectl cluster-info

最新更新