Kubernetes集群VirtualBox网络问题(NAT和仅限主机的适配器)



我正试图在VirtualBox上设置一个kubernetes集群(两个节点,一个主机,一个工作节点)。我的主机运行Windows10,在VirtualBox上我安装了Ubuntu18.10,代码为宇宙。

我在每个VirtualBox上配置了两个适配器,一个NAT和一个仅限主机的适配器。我这样做是因为我需要使用主机IP(NAT)访问一些内部资源,而且我还需要主机和虚拟机之间的稳定网络(仅限主机的网络)。

我已经安装了Kubernetes v1.12.4,并成功地将worker加入到master节点。

NAME                STATUS   ROLES    AGE   VERSION
kubernetes-master   Ready    master   36m   v1.12.4
kubernetes-slave    Ready    <none>   25m   v1.12.4

我正在使用Flannel进行网络连接。

所有吊舱似乎都还好。

NAMESPACE     NAME                                        READY   STATUS    RESTARTS   AGE
default       nginx-server-7bb6997d9c-kdcld               1/1     Running   0          27m
kube-system   coredns-576cbf47c7-btrvb                    1/1     Running   1          38m
kube-system   coredns-576cbf47c7-zfscv                    1/1     Running   1          38m
kube-system   etcd-kubernetes-master                      1/1     Running   1          38m
kube-system   kube-apiserver-kubernetes-master            1/1     Running   1          38m
kube-system   kube-controller-manager-kubernetes-master   1/1     Running   1          38m
kube-system   kube-flannel-ds-amd64-29p96                 1/1     Running   1          28m
kube-system   kube-flannel-ds-amd64-sb2fq                 1/1     Running   1          37m
kube-system   kube-proxy-59v6b                            1/1     Running   1          38m
kube-system   kube-proxy-bfd78                            1/1     Running   0          28m
kube-system   kube-scheduler-kubernetes-master            1/1     Running   1          38m

我已经部署了nginx来验证是否一切正常

NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.96.0.1       <none>        443/TCP   41m
nginx-http   ClusterIP   10.111.151.28   <none>        80/TCP    29m

然而,当我尝试访问nginx时,我得到了一个超时。描述pod给了我以下事件。

Events:
Type     Reason                  Age                 From                       Message
----     ------                  ----                ----                       -------
Normal   Scheduled               32m                 default-scheduler          Successfully assigned default/nginx-server-7bb6997d9c-kdcld to kubernetes-slave
Warning  FailedCreatePodSandBox  32m                 kubelet, kubernetes-slave  Failed create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "dbb2595628fc2579c29779e31e27e27eaeff2dbcf2bdb68467c47f22a3590bd0" network for pod "nginx-server-7bb6997d9c-kdcld": NetworkPlugin cni failed to set up pod "nginx-server-7bb6997d9c-kdcld_default" network: open /run/flannel/subnet.env: no such file or directory
Warning  FailedCreatePodSandBox  32m                 kubelet, kubernetes-slave  Failed create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "801e0f3f8ca4a9b7cc21d87d41141485e1b1da357f2d89e1644acf0ecf634016" network for pod "nginx-server-7bb6997d9c-kdcld": NetworkPlugin cni failed to set up pod "nginx-server-7bb6997d9c-kdcld_default" network: open /run/flannel/subnet.env: no such file or directory
Warning  FailedCreatePodSandBox  32m                 kubelet, kubernetes-slave  Failed create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "77214c757449097bfbe05b24ebb5fd3c7f1d96f7e3e9a3cd48f3b37f30224feb" network for pod "nginx-server-7bb6997d9c-kdcld": NetworkPlugin cni failed to set up pod "nginx-server-7bb6997d9c-kdcld_default" network: open /run/flannel/subnet.env: no such file or directory
Warning  FailedCreatePodSandBox  32m                 kubelet, kubernetes-slave  Failed create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "ebffdd723083d916c0910489e12368dc4069dd99c24a3a4ab1b1d4ab823866ff" network for pod "nginx-server-7bb6997d9c-kdcld": NetworkPlugin cni failed to set up pod "nginx-server-7bb6997d9c-kdcld_default" network: open /run/flannel/subnet.env: no such file or directory
Warning  FailedCreatePodSandBox  32m                 kubelet, kubernetes-slave  Failed create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "d87b93815380246a05470e597a88d50eb31c132a50e30000ab41a456d1e65107" network for pod "nginx-server-7bb6997d9c-kdcld": NetworkPlugin cni failed to set up pod "nginx-server-7bb6997d9c-kdcld_default" network: open /run/flannel/subnet.env: no such file or directory
Warning  FailedCreatePodSandBox  32m                 kubelet, kubernetes-slave  Failed create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "3ef233ef0a6c447134c7b027747a701d6576a80e76c9cc8ffd8287e8ee5f02a4" network for pod "nginx-server-7bb6997d9c-kdcld": NetworkPlugin cni failed to set up pod "nginx-server-7bb6997d9c-kdcld_default" network: open /run/flannel/subnet.env: no such file or directory
Warning  FailedCreatePodSandBox  32m                 kubelet, kubernetes-slave  Failed create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "6b621aab3c57154941b37360240228fe939b528855a5fe8cd9536df63d41ed93" network for pod "nginx-server-7bb6997d9c-kdcld": NetworkPlugin cni failed to set up pod "nginx-server-7bb6997d9c-kdcld_default" network: open /run/flannel/subnet.env: no such file or directory
Warning  FailedCreatePodSandBox  32m                 kubelet, kubernetes-slave  Failed create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "fa992bde90e0a1839180666bedaf74965fb26f3dccb33a66092836a25882ab44" network for pod "nginx-server-7bb6997d9c-kdcld": NetworkPlugin cni failed to set up pod "nginx-server-7bb6997d9c-kdcld_default" network: open /run/flannel/subnet.env: no such file or directory
Warning  FailedCreatePodSandBox  32m                 kubelet, kubernetes-slave  Failed create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "81f74f687e17d67bd2853849f84ece33a118744278d78ac7af3bdeadff8aa9c7" network for pod "nginx-server-7bb6997d9c-kdcld": NetworkPlugin cni failed to set up pod "nginx-server-7bb6997d9c-kdcld_default" network: open /run/flannel/subnet.env: no such file or directory
Warning  FailedCreatePodSandBox  32m (x2 over 32m)   kubelet, kubernetes-slave  (combined from similar events): Failed create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "29188c3e73d08e81b08b2258254dc2691fcaa514ecc96e9df86f2e61ba455b76" network for pod "nginx-server-7bb6997d9c-kdcld": NetworkPlugin cni failed to set up pod "nginx-server-7bb6997d9c-kdcld_default" network: open /run/flannel/subnet.env: no such file or directory
Normal   SandboxChanged          32m (x11 over 32m)  kubelet, kubernetes-slave  Pod sandbox changed, it will be killed and re-created.
Normal   Pulling                 32m                 kubelet, kubernetes-slave  pulling image "nginx"
Normal   Pulled                  32m                 kubelet, kubernetes-slave  Successfully pulled image "nginx"
Normal   Created                 32m                 kubelet, kubernetes-slave  Created container

我尝试过用只配置到虚拟机的桥接适配器进行完全相同的安装,然后一切都如预期的那样工作。

我认为这是一个配置问题,但我无法解决。有人能给我建议吗?

正如我在删除的评论中提到的,我在Ubuntu 18.04主机上重新创建了这个。创建了两个Ubuntu 18.10虚拟机,带有两个适配器(NAT和一个仅限主机的适配器)。我的配置与您在这里指定的配置相同。一切都很好。

我所要做的是手动添加第二个适配器,在节点上运行kubeadm initkubeadm join之前,我使用了netplan

如果您没有这样做,请将仅限主机的适配器网络添加到/etc/netplan/50-cloud-init.yaml中的yaml文件中,然后运行sudo netplan generatesudo netplan apply。对于nginx,我使用了Kubernetes官方文档中的部署。然后我暴露了服务:

kubectl create service nodeport nginx --tcp=80:80从主机在NodePort上卷曲我的节点IP地址可以很好地工作。

这只是为了证明我做了什么,所以它在我的环境中有效。从描述的吊舱错误来看,Flannel本身似乎有问题:

/run/flannel/subnet.env: no such file or directory

我在master上检查了这个目录,它看起来像这样:

/run/frandl/subnet.env

FLANNEL_NETWORK=10.244.0.0/16
FLANNEL_SUBNET=10.244.0.1/24
FLANNEL_MTU=1450
FLANNEL_IPMASQ=true

检查文件是否存在,如果这对您没有帮助,如果您提供更多信息,我们可以尝试进一步进行故障排除。然而,有太多的未知因素,所以我不得不猜测,在某些地方,我的建议是将其全部销毁,然后使用我提供的信息重试,并使用NodePort而不是ClusterIP类型运行nginx。ClusterIP只能从集群内部访问,例如Node。

请让我给这个线程打气。很久以前,我为互联网配置了1个NAT,为SSH远程配置了一个HOST,但错误相同。设置Rancher Longhorn时特别。

现在,我不是那样建的。首先,我使用iptable(1个NAT,1个HOST),在CentOS上构建了网关服务器

然后,其他虚拟机只有1个接口HOST直接连接到GATEWAY SERVER

相关内容

最新更新