尝试使用 kubeadm join 将节点添加到 kubernetes 集群时显示错误

Question

我正在使用 kubeadm 创建一个 kubernetes 集群。Kubeadm init 是成功的。但是当我尝试添加节点时，我看到此错误。任何方向都受到高度赞赏。

kubeadm join 10.127.0.142:6443 --token ddd0 --discovery-token-ca-cert-hash sha256:ddddd
[preflight] running pre-flight checks
[WARNING RequiredIPVSKernelModulesAvailable]: the IPVS proxier will not be used, because the following required kernel modules are not loaded: [ip_vs_sh ip_vs ip_vs_rr ip_vs_wrr] or no builtin kernel ipvs support: map[ip_vs_sh:{} nf_conntrack_ipv4:{} ip_vs:{} ip_vs_rr:{} ip_vs_wrr:{}]
you can solve this problem with following methods:
1. Run 'modprobe -- ' to load missing kernel modules;
2. Provide the missing builtin kernel ipvs support
[discovery] Trying to connect to API Server "10.127.0.142:6443"
[discovery] Created cluster-info discovery client, requesting info from "https://10.127.0.142:6443"
[discovery] Requesting info from "https://10.127.0.142:6443" again to validate TLS against the pinned public key
[discovery] Cluster info signature and contents are valid and TLS certificate validates against pinned roots, will use API Server "10.127.0.142:6443"
[discovery] Successfully established connection with API Server "10.127.0.142:6443"
[kubelet] Downloading configuration for the kubelet from the "kubelet-config-1.12" ConfigMap in the kube-system namespace
configmaps "kubelet-config-1.12" is forbidden: User "system:bootstrap:mq0t2n" cannot get configmaps in the namespace "kube-system"

Answer 1

我很确定您的主节点和工作节点的版本不匹配。
按照此官方说明将集群升级到相同版本。

第二种解决方案是将工作节点降级为主节点版本

Answer 2

自 12 月 12 日发布以来，我开始在 1.12 中看到这种类型的消息，就在 1.13 发布之后。 我使用的是脚本安装，因此我的主节点和工作节点之间没有版本不匹配或任何东西。 如果 1.12 仍然是所需的版本，我发布了针对该权限问题的修复程序：k8s 1.12 kubeadm join 权限修复。

下面还提供了此修复程序：

在主节点上执行步骤 1、2、3、4。

在工作节点上执行步骤 5。

第 1 步：从现有的 "kubelet-config-1.13" ConfigMap 创建一个新的 "kubelet-config-1.12" ConfigMap：

$ kubectl get cm --all-namespaces
$ kubectl -n kube-system get cm kubelet-config-1.13 -o yaml --export > kubelet-config-1.12-cm.yaml
$ vim kubelet-config-1.12-cm.yaml       #modify at the bottom:
#name: kubelet-config-1.12
#delete selfLink
$ kubectl -n kube-system create -f kubelet-config-1.12-cm.yaml

第 2 步：获取令牌前缀：

$ sudo kubeadm token list           #if no output, then create a token:
$ sudo kubeadm token create
TOKEN                       ...     ...
a0b1c2.svn4my9ifft4zxgg     ...     ...
# Token prefix is "a0b1c2"

第 3 步：从现有的 "kubeadm：kubelet-config-1.13" 角色创建一个新的 "kubeadm：kubelet-config-1.12" 角色：

$ kubectl get roles --all-namespaces
$ kubectl -n kube-system get role kubeadm:kubelet-config-1.13 > kubeadm:kubelet-config-1.12-role.yaml
$ vim kubeadm:kubelet-config-1.12-role.yaml    #modify the following:
#name: kubeadm:kubelet-config-1.12
#resourceNames: kubelet-config-1.12
#delete creationTimestamp, resourceVersion, selfLink, uid (because --export option is not supported)    
$ kubectl -n kube-system create -f kubeadm:kubelet-config-1.12-role.yaml

第 4 步：从现有的 "kubeadm：kubelet-config-1.13" 角色绑定创建一个新的角色绑定 "kubeadm：kubelet-config-1.12"：

$ kubectl get rolebindings --all-namespaces
$ kubectl -n kube-system get rolebinding kubeadm:kubelet-config-1.13 > kubeadm:kubelet-config-1.12-rolebinding.yaml
$ vim kubeadm:kubelet-config-1.12-rolebinding.yaml     #modify the following:
#metadata/name: kubeadm:kubelet-config-1.12
#roleRef/name: kubeadm:kubelet-config-1.12
#delete creationTimestamp, resourceVersion, selfLink, uid (because --export option is not supported)
- apiGroup: rbac.authorization.k8s.io                       #add these 3 lines as another group in "subjects:" at the bottom, with the 6 character token prefix from STEP 2
kind: Group
name: system:bootstrap:a0b1c2 
$ kubectl -n kube-system create -f kubeadm:kubelet-config-1.12-rolebinding.yaml

第 5 步：从工作节点运行 kubeadm join：

$ sudo kubeadm join --token <token> <master-IP>:6443 --discovery-token-ca-cert-hash sha256:<key-value> 
# If you receive 2 ERRORS, run kubeadm join again with the following options:
$ sudo kubeadm join --token <token> <master-IP>:6443 --discovery-token-ca-cert-hash sha256:<key-value> --ignore-preflight-errors=FileAvailable--etc-kubernetes-bootstrap-kubelet.conf,FileAvailable--etc-kubernetes-pki-ca.crt

Answer 3

1. kubectl -n kube-system get role kubeadm:kubelet-config-1.13 > kubeadm:kubelet-config-1.12-role.yaml

   #metadata/name: kubeadm:kubelet-config-1.12
   #roleRef/name: kubeadm:kubelet-config-1.12
   #delete creationTimestamp, resourceVersion, selfLink, uid (because --export option is not supported)
   

2. kubectl apply -f kubeadm:kubelet-config-1.12-role.yaml

3. kubectl -n kube-system get rolebinding kubeadm:kubelet-config-1.13 > kubeadm:kubelet-config-1.12-rolebinding.yaml

   #metadata/name: kubeadm:kubelet-config-1.12
   #roleRef/name: kubeadm:kubelet-config-1.12
   #delete creationTimestamp, resourceVersion, selfLink, uid (because --export option is not supported)
   

4. kubectl apply -f kubeadm:kubelet-config-1.12-rolebinding.yaml

5. kubectl get configmap kubelet-config-1.13 -n kube-system -oyaml > kubelet-config-1.12

   #metadata/name: kubelet-config-1.12
   #roleRef/name: kubelet-config-1.12
   #delete creationTimestamp, resourceVersion, selfLink, uid (because --export option is not supported)
   

6. kubectl apply -f kubelet-config-1.12

7. 登录到要加入的节点并删除以下文件：

   rm /etc/kubernetes/bootstrap-kubelet.conf
   rm /etc/kubernetes/pki/ca.crt
   

8. 现在运行kubeadm联接命令