解决问题
我按照 kubernetes-the-hard-way 指南设置了一个单节点 kubernetes，除了我在 CentOS-7 上运行，并且在同一节点中部署了一个主节点和一个工作节点。我已经关闭了防火墙服务。
安装后，我部署了一个 mongodb 服务，但是无法访问集群 IP，但可以访问端点。服务详情如下：
$ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.254.0.1 <none> 443/TCP 2m
mongodb ClusterIP 10.254.0.117 <none> 27017/TCP 55s
$ kubectl describe svc mongodb
Name: mongodb
Namespace: default
Labels: io.kompose.service=mongodb
Annotations: kompose.cmd=kompose convert -f docker-compose.yml
kompose.version=1.11.0 (39ad614)
kubectl.kubernetes.io/last-applied-configuration=
{"apiVersion":"v1","kind":"Service","metadata":{"annotations":
{"kompose.cmd":"kompose convert -f docker-compose.yml","kompose.version":"1.11.0
(39ad614...
Selector: io.kompose.service=mongodb
Type: ClusterIP
IP: 10.254.0.117
Port: 27017 27017/TCP
TargetPort: 27017/TCP
Endpoints: 10.254.0.2:27017
Session Affinity: None
Events: <none>
它可以工作，但是当我运行 mongo 10.254.0.117 时，它不起作用。顺便说一句，例如，如果我启动另一个 mongo pod
kubectl run mongo-shell -ti --image=mongo --restart=Never bash
我尝试了 mongo 10.254.0.2 和 mongo 10.254.0.117，它们根本不起作用。
我使用的 kubernetes 版本是 1.10.0。
我认为这是一个 kube 代理问题，kube 代理配置如下：
[Unit]
Description=Kubernetes Kube-Proxy Server
Documentation=https://kubernetes.io/docs/concepts/overview/components/#kube-
proxy https://kubernetes.io/docs/reference/generated/kube-proxy/
After=network.target
[Service]
ExecStart=/usr/local/bin/kube-proxy
--config=/var/lib/kubelet/kube-proxy-config.yaml
--logtostderr=true
--v=2
Restart=on-failure
RestartSec=5
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
配置文件是
kind: KubeProxyConfiguration
apiVersion: kubeproxy.config.k8s.io/v1alpha1
clientConnection:
kubeconfig: "/var/lib/kubelet/kube-proxy.kubeconfig"
mode: "iptables"
clusterCIDR: "10.254.0.0/16"
这是我得到的 ip 表
sudo iptables -t nat -nL
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes service portals */
DOCKER all -- 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes service portals */
DOCKER all -- 0.0.0.0/0 !127.0.0.0/8 ADDRTYPE match dst-type LOCAL
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
KUBE-POSTROUTING all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes postrouting rules */
MASQUERADE all -- 172.17.0.0/16 0.0.0.0/0
CNI-0f56c935ec75c77eb189a5fe all -- 10.254.0.0/24 0.0.0.0/0 /* name: "bridge" id: "a54a2f20dbe5d24ec4fb6b059f23aae392cc26853cf2b474a56dff2a2f2d6bb6" */
CNI-d2a650ff06e253010ea31f3d all -- 10.254.0.0/24 0.0.0.0/0 /* name: "bridge" id: "f3252d60a15faa5ff6c4b2aabebdb47aa5652e12c9d874f538b33d6c5913ba47" */
CNI-34b02c799f7bc4e979c15266 all -- 10.254.0.0/24 0.0.0.0/0 /* name: "bridge" id: "5a87d86a62dd299e1d36b2ccd631d58896f2724ad9b4e14a93b9dfaa162b09e3" */
CNI-eb80e2736e1009010a27b4b4 all -- 10.254.0.0/24 0.0.0.0/0 /* name: "bridge" id: "1891a61e27b764e4a36717166a2b83ce7d2baa5258e54f0ea183c4433b04de38" */
CNI-4d1b80b0072ade1be68c43d1 all -- 10.254.0.0/24 0.0.0.0/0 /* name: "bridge" id: "2b90e720350fa78bf6e6756b941526bf181e0b48c6b87207bbc8f097933e67ba" */
CNI-7699fcd0ab82a702bac28bc9 all -- 10.254.0.0/24 0.0.0.0/0 /* name: "bridge" id: "3feed2ec479bd17f82cac60adfd1c79c81d4c53d536daa74a46e05f462e2d895" */
CNI-871343dd2a1a9738c94b4dba all -- 10.254.0.0/24 0.0.0.0/0 /* name: "bridge" id: "1a3a7b27889e54494d1e9699efb158dc8f3bb85b147b80db84038c07fd4c9910" */
CNI-3c0d02d02e5aa29b38ada7ba all -- 10.254.0.0/24 0.0.0.0/0 /* name: "bridge" id: "cdd5d6cf1a772b2acd37471046f53d0aa635733f0d5447a11d76dbb2ee216378" */
Chain CNI-0f56c935ec75c77eb189a5fe (1 references)
target prot opt source destination
ACCEPT all -- 0.0.0.0/0 10.254.0.0/24 /* name: "bridge" id: "a54a2f20dbe5d24ec4fb6b059f23aae392cc26853cf2b474a56dff2a2f2d6bb6" */
MASQUERADE all -- 0.0.0.0/0 !224.0.0.0/4 /* name: "bridge" id: "a54a2f20dbe5d24ec4fb6b059f23aae392cc26853cf2b474a56dff2a2f2d6bb6" */
Chain CNI-34b02c799f7bc4e979c15266 (1 references)
target prot opt source destination
ACCEPT all -- 0.0.0.0/0 10.254.0.0/24 /* name: "bridge" id: "5a87d86a62dd299e1d36b2ccd631d58896f2724ad9b4e14a93b9dfaa162b09e3" */
MASQUERADE all -- 0.0.0.0/0 !224.0.0.0/4 /* name: "bridge" id: "5a87d86a62dd299e1d36b2ccd631d58896f2724ad9b4e14a93b9dfaa162b09e3" */
Chain CNI-3c0d02d02e5aa29b38ada7ba (1 references)
target prot opt source destination
ACCEPT all -- 0.0.0.0/0 10.254.0.0/24 /* name: "bridge" id: "cdd5d6cf1a772b2acd37471046f53d0aa635733f0d5447a11d76dbb2ee216378" */
MASQUERADE all -- 0.0.0.0/0 !224.0.0.0/4 /* name: "bridge" id: "cdd5d6cf1a772b2acd37471046f53d0aa635733f0d5447a11d76dbb2ee216378" */
Chain CNI-4d1b80b0072ade1be68c43d1 (1 references)
target prot opt source destination
ACCEPT all -- 0.0.0.0/0 10.254.0.0/24 /* name: "bridge" id: "2b90e720350fa78bf6e6756b941526bf181e0b48c6b87207bbc8f097933e67ba" */
MASQUERADE all -- 0.0.0.0/0 !224.0.0.0/4 /* name: "bridge" id: "2b90e720350fa78bf6e6756b941526bf181e0b48c6b87207bbc8f097933e67ba" */
Chain CNI-7699fcd0ab82a702bac28bc9 (1 references)
target prot opt source destination
ACCEPT all -- 0.0.0.0/0 10.254.0.0/24 /* name: "bridge" id: "3feed2ec479bd17f82cac60adfd1c79c81d4c53d536daa74a46e05f462e2d895" */
MASQUERADE all -- 0.0.0.0/0 !224.0.0.0/4 /* name: "bridge" id: "3feed2ec479bd17f82cac60adfd1c79c81d4c53d536daa74a46e05f462e2d895" */
Chain CNI-871343dd2a1a9738c94b4dba (1 references)
target prot opt source destination
ACCEPT all -- 0.0.0.0/0 10.254.0.0/24 /* name: "bridge" id: "1a3a7b27889e54494d1e9699efb158dc8f3bb85b147b80db84038c07fd4c9910" */
MASQUERADE all -- 0.0.0.0/0 !224.0.0.0/4 /* name: "bridge" id: "1a3a7b27889e54494d1e9699efb158dc8f3bb85b147b80db84038c07fd4c9910" */
Chain CNI-d2a650ff06e253010ea31f3d (1 references)
target prot opt source destination
ACCEPT all -- 0.0.0.0/0 10.254.0.0/24 /* name: "bridge" id: "f3252d60a15faa5ff6c4b2aabebdb47aa5652e12c9d874f538b33d6c5913ba47" */
MASQUERADE all -- 0.0.0.0/0 !224.0.0.0/4 /* name: "bridge" id: "f3252d60a15faa5ff6c4b2aabebdb47aa5652e12c9d874f538b33d6c5913ba47" */
Chain CNI-eb80e2736e1009010a27b4b4 (1 references)
target prot opt source destination
ACCEPT all -- 0.0.0.0/0 10.254.0.0/24 /* name: "bridge" id: "1891a61e27b764e4a36717166a2b83ce7d2baa5258e54f0ea183c4433b04de38" */
MASQUERADE all -- 0.0.0.0/0 !224.0.0.0/4 /* name: "bridge" id: "1891a61e27b764e4a36717166a2b83ce7d2baa5258e54f0ea183c4433b04de38" */
Chain DOCKER (2 references)
target prot opt source destination
RETURN all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-MARK-DROP (0 references)
target prot opt source destination
MARK all -- 0.0.0.0/0 0.0.0.0/0 MARK or 0x8000
Chain KUBE-MARK-MASQ (4 references)
target prot opt source destination
MARK all -- 0.0.0.0/0 0.0.0.0/0 MARK or 0x4000
Chain KUBE-NODEPORTS (1 references)
target prot opt source destination
Chain KUBE-POSTROUTING (1 references)
target prot opt source destination
MASQUERADE all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes service traffic requiring SNAT */ mark match 0x4000/0x4000
Chain KUBE-SEP-G5V522HWZT6RKRAC (2 references)
target prot opt source destination
KUBE-MARK-MASQ all -- 192.168.56.3 0.0.0.0/0 /* default/kubernetes:https */
DNAT tcp -- 0.0.0.0/0 0.0.0.0/0 /* default/kubernetes:https */ recent: SET name: KUBE-SEP-G5V522HWZT6RKRAC side: source mask: 255.255.255.255 tcp to:192.168.56.3:6443
Chain KUBE-SEP-O34O4OGFBAADOMEG (1 references)
target prot opt source destination
KUBE-MARK-MASQ all -- 10.254.0.2 0.0.0.0/0 /* default/mongodb:27017 */
DNAT tcp -- 0.0.0.0/0 0.0.0.0/0 /* default/mongodb:27017 */ tcp to:10.254.0.2:27017
Chain KUBE-SERVICES (2 references)
target prot opt source destination
KUBE-MARK-MASQ tcp -- !10.254.0.0/16 10.254.0.1 /* default/kubernetes:https cluster IP */ tcp dpt:443
KUBE-SVC-NPX46M4PTMTKRN6Y tcp -- 0.0.0.0/0 10.254.0.1 /* default/kubernetes:https cluster IP */ tcp dpt:443
KUBE-MARK-MASQ tcp -- !10.254.0.0/16 10.254.0.117 /* default/mongodb:27017 cluster IP */ tcp dpt:27017
KUBE-SVC-ZDG6MRTNE2LQFT34 tcp -- 0.0.0.0/0 10.254.0.117 /* default/mongodb:27017 cluster IP */ tcp dpt:27017
KUBE-NODEPORTS all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes service nodeports; NOTE: this must be the last rule in this chain */ ADDRTYPE match dst-type LOCAL
Chain KUBE-SVC-NPX46M4PTMTKRN6Y (1 references)
target prot opt source destination
KUBE-SEP-G5V522HWZT6RKRAC all -- 0.0.0.0/0 0.0.0.0/0 /* default/kubernetes:https */ recent: CHECK seconds: 10800 reap name: KUBE-SEP-G5V522HWZT6RKRAC side: source mask: 255.255.255.255
KUBE-SEP-G5V522HWZT6RKRAC all -- 0.0.0.0/0 0.0.0.0/0 /* default/kubernetes:https */
Chain KUBE-SVC-ZDG6MRTNE2LQFT34 (1 references)
target prot opt source destination
KUBE-SEP-O34O4OGFBAADOMEG all -- 0.0.0.0/0 0.0.0.0/0 /* default/mongodb:27017 */
我删除了 kubelet 服务的 --network-plugin=cni 标志并将 kubernetes 升级到 1.13.0