我在裸机上安装了一个Kubernetes集群(使用VMware虚拟机)，具有以下节点
master-01 Ready control-plane,master 5d3h v1.21.3
master-02 Ready control-plane,master 5d3h v1.21.3
master-03 Ready control-plane,master 5d3h v1.21.3
worker-01 Ready <none> 5d2h v1.21.3
worker-02 Ready <none> 5d2h v1.21.3
worker-03 Ready <none> 5d2h v1.21.3
Metallb安装为集群的负载均衡器，calico安装为CNI
我还安装了nginx-ingress-controller with helm
$ helm repo add nginx-stable https://helm.nginx.com/stable
$ helm repo update
$ helm install ingress-controller nginx-stable/nginx-ingress
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
labels:
app: nginx-app
spec:
replicas: 2
selector:
matchLabels:
app: nginx-app
template:
metadata:
labels:
app: nginx-app
spec:
containers:
- name: nginx
image: nginx
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: nginx-service
spec:
selector:
app: nginx-app
#type: LoadBalancer
ports:
- protocol: TCP
port: 80
targetPort: 80
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-myapp
annotations:
# use the shared ingress-nginx
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: myapp.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: nginx-service
port:
number: 80
我使用负载均衡器类型的部署从metallb获得IP并且工作正常，但是当我添加入口时，尽管分配了IP，但我得到错误502坏网关，如下所示:
防火墙已启用，但所需的端口已打开
6443/tcp 2379-2380/tcp 10250-10252/tcp 179/tcp 7946/tcp 7946/udp 8443/tcp on master nodes
10250/tcp 30000-32767/tcp 7946/tcp 7946/udp 8443/tcp 179/tcp on worker nodes
我的服务和pod工作正常
kubectl get svc -o wide
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
ingress-controller-nginx-ingress LoadBalancer 10.101.17.180 10.1.210.100 80:31509/TCP,443:30004/TCP 33m app=ingress-controller-nginx-ingress
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 6d <none>
nginx-service ClusterIP 10.101.48.198 <none> 80/TCP 31m app=nginx-app
我的入口日志给我错误，没有路由到内部IP
2021/07/29 07:46:24 [error] 42#42: *8 connect() failed (113: No route to host) while connecting to upstream, client: 10.1.210.5, server: myapp.com, request: "GET / HTTP/1.1", upstream: "http://192.168.171.17:80/", host: "myapp.com"
10.1.210.5 - - [29/Jul/2021:07:46:24 +0000] "GET / HTTP/1.1" 502 157 "-" "curl/7.68.0" "-"
W0729 07:50:16.416830 1 warnings.go:70] networking.k8s.io/v1beta1 Ingress is deprecated in v1.19+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
192.168.2.131 - - [29/Jul/2021:07:51:03 +0000] "GET / HTTP/1.1" 404 555 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.107 Safari/537.36" "-"
192.168.2.131 - - [29/Jul/2021:07:51:03 +0000] "GET /favicon.ico HTTP/1.1" 404 555 "http://10.1.210.100/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.107 Safari/537.36" "-"
W0729 07:56:43.420282 1 warnings.go:70] networking.k8s.io/v1beta1 Ingress is deprecated in v1.19+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
W0729 08:05:28.422594 1 warnings.go:70] networking.k8s.io/v1beta1 Ingress is deprecated in v1.19+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
W0729 08:10:45.425329 1 warnings.go:70] networking.k8s.io/v1beta1 Ingress is deprecated in v1.19+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
2021/07/29 08:13:59 [error] 42#42: *12 connect() failed (113: No route to host) while connecting to upstream, client: 10.1.210.5, server: myapp.com, request: "GET / HTTP/1.1", upstream: "http://192.168.171.17:80/", host: "myapp.com"
10.1.210.5 - - [29/Jul/2021:08:13:59 +0000] "GET / HTTP/1.1" 502 157 "-" "curl/7.68.0" "-"
2021/07/29 08:14:09 [error] 42#42: *14 connect() failed (113: No route to host) while connecting to upstream, client: 10.1.210.5, server: myapp.com, request: "GET / HTTP/1.1", upstream: "http://192.168.171.17:80/", host: "myapp.com"
10.1.210.5 - - [29/Jul/2021:08:14:09 +0000] "GET / HTTP/1.1" 502 157 "-" "curl/7.68.0" "-"
有什么想法吗?
编辑:正如这里询问的服务和pod的描述
$ kubectl describe pod nginx-deployment-6f7d8d4d55-sncdr
Name: nginx-deployment-6f7d8d4d55-sncdr
Namespace: default
Priority: 0
Node: worker-01/10.1.210.63
Start Time: Thu, 29 Jul 2021 08:43:59 +0100
Labels: app=nginx-app
pod-template-hash=6f7d8d4d55
Annotations: cni.projectcalico.org/podIP: 192.168.171.17/32
cni.projectcalico.org/podIPs: 192.168.171.17/32
Status: Running
IP: 192.168.171.17
IPs:
IP: 192.168.171.17
Controlled By: ReplicaSet/nginx-deployment-6f7d8d4d55
Containers:
nginx:
Container ID: docker://fc61b73f8a833ad13b8956d8ce151b221b75a58a9a2fbae928464f3b0a77cca2
Image: nginx
Image ID: docker-pullable://nginx@sha256:8f335768880da6baf72b70c701002b45f4932acae8d574dedfddaf967fc3ac90
Port: 80/TCP
Host Port: 0/TCP
State: Running
Started: Thu, 29 Jul 2021 08:44:01 +0100
Ready: True
Restart Count: 0
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-wkc48 (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
kube-api-access-wkc48:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 16m default-scheduler Successfully assigned default/nginx-deployment-6f7d8d4d55-sncdr to worker-01
Normal Pulling 16m kubelet Pulling image "nginx"
Normal Pulled 16m kubelet Successfully pulled image "nginx" in 1.51808376s
Normal Created 16m kubelet Created container nginx
Normal Started 16m kubelet Started container nginx
$ kubectl describe svc ingress-controller-nginx-ingress
Name: ingress-controller-nginx-ingress
Namespace: default
Labels: app.kubernetes.io/instance=ingress-controller
app.kubernetes.io/managed-by=Helm
app.kubernetes.io/name=ingress-controller-nginx-ingress
helm.sh/chart=nginx-ingress-0.10.0
Annotations: meta.helm.sh/release-name: ingress-controller
meta.helm.sh/release-namespace: default
Selector: app=ingress-controller-nginx-ingress
Type: LoadBalancer
IP Family Policy: SingleStack
IP Families: IPv4
IP: 10.101.17.180
IPs: 10.101.17.180
LoadBalancer Ingress: 10.1.210.100
Port: http 80/TCP
TargetPort: 80/TCP
NodePort: http 31509/TCP
Endpoints: 192.168.37.202:80
Port: https 443/TCP
TargetPort: 443/TCP
NodePort: https 30004/TCP
Endpoints: 192.168.37.202:443
Session Affinity: None
External Traffic Policy: Local
HealthCheck NodePort: 31108
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal IPAllocated 18m metallb-controller Assigned IP "10.1.210.100"
Normal nodeAssigned 3m21s (x182 over 18m) metallb-speaker announcing from node "worker-02"
$ kubectl describe svc nginx-service
Name: nginx-service
Namespace: default
Labels: <none>
Annotations: <none>
Selector: app=nginx-app
Type: ClusterIP
IP Family Policy: SingleStack
IP Families: IPv4
IP: 10.101.48.198
IPs: 10.101.48.198
Port: <unset> 80/TCP
TargetPort: 80/TCP
Endpoints: 192.168.171.17:80
Session Affinity: None
Events: <none>
$ kubectl exec -it ingress-controller-nginx-ingress-dd5db86dc-gqdpm -- /bin/bash
nginx@ingress-controller-nginx-ingress-dd5db86dc-gqdpm:/$ curl 192.168.171.17:80
curl: (7) Failed to connect to 192.168.171.17 port 80: No route to host
nginx@ingress-controller-nginx-ingress-dd5db86dc-gqdpm:/$ curl 192.168.171.17
curl: (7) Failed to connect to 192.168.171.17 port 80: No route to host
nginx@ingress-controller-nginx-ingress-dd5db86dc-gqdpm:/$ curl 10.101.48.198
curl: (7) Failed to connect to 10.101.48.198 port 80: Connection timed out
nginx@ingress-controller-nginx-ingress-dd5db86dc-gqdpm:/$ curl nginx-deployment-6f7d8d4d55-sncdr
curl: (6) Could not resolve host: nginx-deployment-6f7d8d4d55-sncdr
nginx@ingress-controller-nginx-ingress-dd5db86dc-gqdpm:/$
老实说，我不明白为什么curl svcip不再工作了;昨天还行。
问题是防火墙问题，我禁用了防火墙，它现在工作，我认为必须打开端口8443，但它似乎是另一个端口，如果有人能告诉我哪一个
谢谢
我在k3s中遇到了类似的问题。我在防火墙中启用了伪装firewall-cmd --permanent --add-masquerade && firewall-cmd --reload
这篇文章的想法:https://github.com/k3s-io/k3s/issues/1646#issuecomment-881191877