Metallb LoadBalancer is stuck on pending



我正在尝试设置一个裸机Kubernetes服务器,包括metallbLoadBalancer。因此,我按照同类项目https://kind.sigs.k8s.io/docs/user/loadbalancer/的网站上发布的说明来做。为了测试我的安装,我尝试部署默认的nginx网络服务器,并通过浏览器到达它。不幸的是,创建的LoadBalancer类型的服务卡在EXTERNAL IP <Pending>上。我会附上尽可能多的资料。我希望有人能帮我解决这个问题。如果你需要更多的信息,请告诉我。

我是这样做的:

  1. 根据https://kind.sigs.k8s.io/docs/user/loadbalancer/
  2. 安装metallb LoadBalancer通过kubectl create deployment nginx --image nginx创建nginx部署
  3. 创建相应的业务kubectl expose deployment/nginx --type="LoadBalancer" --port 8080

输出

版本与节点

$ kubectl version --short
Client Version: v1.20.2
Server Version: v1.20.2
$ kubectl get nodes -o wide
NAME      STATUS   ROLES                  AGE     VERSION   INTERNAL-IP     EXTERNAL-IP   OS-IMAGE       KERNEL-VERSION     CONTAINER-RUNTIME
gemini    Ready    control-plane,master   3d      v1.20.2   192.168.2.203   <none>        Ubuntu 20.10   5.8.0-1013-raspi   docker://19.3.13
phoenix   Ready    <none>                 2d23h   v1.20.2   192.168.2.129   <none>        Ubuntu 20.10   5.8.0-1013-raspi   docker://19.3.13
taurus    Ready    <none>                 2d23h   v1.20.2   192.168.2.201   <none>        Ubuntu 20.10   5.8.0-1013-raspi   docker://19.3.13
virgo     Ready    <none>                 2d23h   v1.20.2   192.168.2.202   <none>        Ubuntu 20.10   5.8.0-1013-raspi   docker://19.3.13

所有名称空间
$ kubectl get all --all-namespaces
NAMESPACE              NAME                                             READY   STATUS             RESTARTS   AGE
default                pod/nginx-6799fc88d8-62cjd                       1/1     Running            0          59m
kube-system            pod/calico-kube-controllers-86bddfcff-ccrhg      1/1     Running            5          3d
kube-system            pod/calico-node-djt5s                            0/1     CrashLoopBackOff   1007       3d
kube-system            pod/calico-node-jddnl                            1/1     Running            3          3d
kube-system            pod/calico-node-nxwlw                            1/1     Running            5          3d
kube-system            pod/calico-node-zrxzl                            1/1     Running            3          3d
kube-system            pod/coredns-74ff55c5b-kb2nm                      1/1     Running            5          3d1h
kube-system            pod/coredns-74ff55c5b-wsgs5                      1/1     Running            5          3d1h
kube-system            pod/etcd-gemini                                  1/1     Running            6          3d1h
kube-system            pod/kube-apiserver-gemini                        1/1     Running            6          3d1h
kube-system            pod/kube-controller-manager-gemini               1/1     Running            6          3d1h
kube-system            pod/kube-proxy-7fcjz                             1/1     Running            6          3d1h
kube-system            pod/kube-proxy-84rr7                             1/1     Running            3          3d
kube-system            pod/kube-proxy-lc88w                             1/1     Running            3          3d
kube-system            pod/kube-proxy-v4qd9                             1/1     Running            3          3d
kube-system            pod/kube-scheduler-gemini                        1/1     Running            6          3d1h
kubernetes-dashboard   pod/dashboard-metrics-scraper-79c5968bdc-mlb4s   1/1     Running            5          3d
kubernetes-dashboard   pod/kubernetes-dashboard-7448ffc97b-nq5c9        1/1     Running            5          3d
metallb-system         pod/controller-5c797bc4f8-2zx7l                  1/1     Running            0          101m
metallb-system         pod/speaker-5kxc9                                1/1     Running            0          101m
metallb-system         pod/speaker-b6lbh                                1/1     Running            0          101m
metallb-system         pod/speaker-c8sr7                                1/1     Running            0          101m
metallb-system         pod/speaker-fwbgt                                1/1     Running            1          101m
NAMESPACE              NAME                                TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)                  AGE
default                service/kubernetes                  ClusterIP      10.96.0.1        <none>        443/TCP                  23h
default                service/nginx                       LoadBalancer   10.110.25.14     <pending>     8080:30444/TCP           24m
kube-system            service/calico-etcd                 ClusterIP      10.96.232.136    <none>        6666/TCP                 3d
kube-system            service/calico-typha                ClusterIP      10.109.108.233   <none>        5473/TCP                 3d
kube-system            service/kube-dns                    ClusterIP      10.96.0.10       <none>        53/UDP,53/TCP,9153/TCP   3d1h
kubernetes-dashboard   service/dashboard-metrics-scraper   ClusterIP      10.110.70.52     <none>        8000/TCP                 3d
kubernetes-dashboard   service/kubernetes-dashboard        NodePort       10.106.194.127   <none>        443:31741/TCP            3d
NAMESPACE        NAME                         DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE
kube-system      daemonset.apps/calico-node   4         4         3       4            3           kubernetes.io/os=linux   3d
kube-system      daemonset.apps/kube-proxy    4         4         4       4            4           kubernetes.io/os=linux   3d1h
metallb-system   daemonset.apps/speaker       4         4         4       4            4           kubernetes.io/os=linux   101m
NAMESPACE              NAME                                        READY   UP-TO-DATE   AVAILABLE   AGE
default                deployment.apps/nginx                       1/1     1            1           59m
kube-system            deployment.apps/calico-kube-controllers     1/1     1            1           3d
kube-system            deployment.apps/coredns                     2/2     2            2           3d1h
kubernetes-dashboard   deployment.apps/dashboard-metrics-scraper   1/1     1            1           3d
kubernetes-dashboard   deployment.apps/kubernetes-dashboard        1/1     1            1           3d
metallb-system         deployment.apps/controller                  1/1     1            1           101m
NAMESPACE              NAME                                                   DESIRED   CURRENT   READY   AGE
default                replicaset.apps/nginx-6799fc88d8                       1         1         1       59m
kube-system            replicaset.apps/calico-kube-controllers-56b44cd6d5     0         0         0       3d
kube-system            replicaset.apps/calico-kube-controllers-86bddfcff      1         1         1       3d
kube-system            replicaset.apps/coredns-74ff55c5b                      2         2         2       3d1h
kubernetes-dashboard   replicaset.apps/dashboard-metrics-scraper-79c5968bdc   1         1         1       3d
kubernetes-dashboard   replicaset.apps/kubernetes-dashboard-7448ffc97b        1         1         1       3d
metallb-system         replicaset.apps/controller-5c797bc4f8                  1         1         1       101m

默认名称空间:

$ kubectl get all
NAME                         READY   STATUS    RESTARTS   AGE
pod/nginx-6799fc88d8-62cjd   1/1     Running   0          29m
NAME                 TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
service/kubernetes   ClusterIP      10.96.0.1        <none>        443/TCP          23h
service/nginx        LoadBalancer   10.110.25.14     <pending>     8080:30444/TCP   29m
NAME                    READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/nginx   1/1     1            1           29m
NAME                               DESIRED   CURRENT   READY   AGE
replicaset.apps/nginx-6799fc88d8   1         1         1       29m

名称空间metallb-system:

$ kubectl get all -n metallb-system
NAME                              READY   STATUS    RESTARTS   AGE
pod/controller-5c797bc4f8-2zx7l   1/1     Running   0          73m
pod/speaker-5kxc9                 1/1     Running   0          73m
pod/speaker-b6lbh                 1/1     Running   0          73m
pod/speaker-c8sr7                 1/1     Running   0          73m
pod/speaker-fwbgt                 1/1     Running   1          73m
NAME                     DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE
daemonset.apps/speaker   4         4         4       4            4           kubernetes.io/os=linux   73m
NAME                         READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/controller   1/1     1            1           73m
NAME                                    DESIRED   CURRENT   READY   AGE
replicaset.apps/controller-5c797bc4f8   1         1         1       73m

配置地图
$ kubectl describe configmap config -n metallb-system
Name:         config
Namespace:    metallb-system
Labels:       <none>
Annotations:  <none>
Data
====
config:
----
address-pools:
- name: default
protocol: layer2
addresses:
- 192.168.2.210-192.168.2.250
Events:  <none>

Kube-proxy

$ kubectl describe configmap -n kube-system kube-proxy
Name:         kube-proxy
Namespace:    kube-system
Labels:       app=kube-proxy
Annotations:  kubeadm.kubernetes.io/component-config.hash: sha256:038b6c93b0ae1da89eb9c0c589c8e3439e9c91b849975435c29f905041b9b9fc
Data
====
config.conf:
----
apiVersion: kubeproxy.config.k8s.io/v1alpha1
bindAddress: 0.0.0.0
bindAddressHardFail: false
clientConnection:
acceptContentTypes: ""
burst: 0
contentType: ""
kubeconfig: /var/lib/kube-proxy/kubeconfig.conf
qps: 0
clusterCIDR: 192.168.0.0/16
configSyncPeriod: 0s
conntrack:
maxPerCore: null
min: null
tcpCloseWaitTimeout: null
tcpEstablishedTimeout: null
detectLocalMode: ""
enableProfiling: false
healthzBindAddress: ""
hostnameOverride: ""
iptables:
masqueradeAll: false
masqueradeBit: null
minSyncPeriod: 0s
syncPeriod: 0s
ipvs:
excludeCIDRs: null
minSyncPeriod: 0s
scheduler: ""
strictARP: true
syncPeriod: 0s
tcpFinTimeout: 0s
tcpTimeout: 0s
udpTimeout: 0s
kind: KubeProxyConfiguration
metricsBindAddress: ""
mode: ""
nodePortAddresses: null
oomScoreAdj: null
portRange: ""
showHiddenMetricsForVersion: ""
udpIdleTimeout: 0s
winkernel:
enableDSR: false
networkName: ""
sourceVip: ""
kubeconfig.conf:
----
apiVersion: v1
kind: Config
clusters:
- cluster:
certificate-authority: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
server: https://192.168.2.203:6443
name: default
contexts:
- context:
cluster: default
namespace: default
user: default
name: default
current-context: default
users:
- name: default
user:
tokenFile: /var/run/secrets/kubernetes.io/serviceaccount/token
Events:  <none>

Metallb控制器和扬声器pod之一

$ kubectl describe pods -n metallb-system
Name:         controller-5c797bc4f8-2zx7l
Namespace:    metallb-system
Priority:     0
Node:         virgo/192.168.2.202
Start Time:   Tue, 09 Feb 2021 14:10:16 +0000
Labels:       app=metallb
component=controller
pod-template-hash=5c797bc4f8
Annotations:  cni.projectcalico.org/podIP: 192.168.9.246/32
cni.projectcalico.org/podIPs: 192.168.9.246/32
prometheus.io/port: 7472
prometheus.io/scrape: true
Status:       Running
IP:           192.168.9.246
IPs:
IP:           192.168.9.246
Controlled By:  ReplicaSet/controller-5c797bc4f8
Containers:
controller:
Container ID:  docker://b10ee06c88facfc97fcd246bcd214c746deea60073dc8928f478c89ca3a4feb9
Image:         quay.io/metallb/controller:main
Image ID:      docker-pullable://quay.io/metallb/controller@sha256:1e1a1bd75cebcecfe84db28c9a8605caeac3e30e57905185822c79fc00e5dcaf
Port:          7472/TCP
Host Port:     0/TCP
Args:
--port=7472
--config=config
State:          Running
Started:      Tue, 09 Feb 2021 14:10:43 +0000
Ready:          True
Restart Count:  0
Limits:
cpu:     100m
memory:  100Mi
Requests:
cpu:        100m
memory:     100Mi
Environment:  <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from controller-token-ghmtk (ro)
Conditions:
Type              Status
Initialized       True
Ready             True
ContainersReady   True
PodScheduled      True
Volumes:
controller-token-ghmtk:
Type:        Secret (a volume populated by a Secret)
SecretName:  controller-token-ghmtk
Optional:    false
QoS Class:       Guaranteed
Node-Selectors:  kubernetes.io/os=linux
Tolerations:     node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type    Reason     Age   From               Message
----    ------     ----  ----               -------
Normal  Scheduled  89m   default-scheduler  Successfully assigned metallb-system/controller-5c797bc4f8-2zx7l to virgo
Normal  Pulling    89m   kubelet            Pulling image "quay.io/metallb/controller:main"
Normal  Pulled     89m   kubelet            Successfully pulled image "quay.io/metallb/controller:main" in 23.412886883s
Normal  Created    89m   kubelet            Created container controller
Normal  Started    89m   kubelet            Started container controller

Name:         speaker-5kxc9
Namespace:    metallb-system
Priority:     0
Node:         taurus/192.168.2.201
Start Time:   Tue, 09 Feb 2021 14:10:16 +0000
Labels:       app=metallb
component=speaker
controller-revision-hash=7548d88c64
pod-template-generation=1
Annotations:  prometheus.io/port: 7472
prometheus.io/scrape: true
Status:       Running
IP:           192.168.2.201
IPs:
IP:           192.168.2.201
Controlled By:  DaemonSet/speaker
Containers:
speaker:
Container ID:  docker://e5ff811c47191f67c7089ce2464abff4fb8e4d3084f8053eb591c9d21e9a2276
Image:         quay.io/metallb/speaker:main
Image ID:      docker-pullable://quay.io/metallb/speaker@sha256:90a8cbdd543870e7db841a49743f3d8cd72b7074b5299086e09c5c1d92331eb4
Port:          7472/TCP
Host Port:     7472/TCP
Args:
--port=7472
--config=config
State:          Running
Started:      Tue, 09 Feb 2021 14:10:33 +0000
Ready:          True
Restart Count:  0
Limits:
cpu:     100m
memory:  100Mi
Requests:
cpu:     100m
memory:  100Mi
Environment:
METALLB_NODE_NAME:       (v1:spec.nodeName)
METALLB_HOST:            (v1:status.hostIP)
METALLB_ML_BIND_ADDR:    (v1:status.podIP)
METALLB_ML_LABELS:      app=metallb,component=speaker
METALLB_ML_NAMESPACE:   metallb-system (v1:metadata.namespace)
METALLB_ML_SECRET_KEY:  <set to the key 'secretkey' in secret 'memberlist'>  Optional: false
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from speaker-token-xfz9p (ro)
Conditions:
Type              Status
Initialized       True
Ready             True
ContainersReady   True
PodScheduled      True
Volumes:
speaker-token-xfz9p:
Type:        Secret (a volume populated by a Secret)
SecretName:  speaker-token-xfz9p
Optional:    false
QoS Class:       Guaranteed
Node-Selectors:  kubernetes.io/os=linux
Tolerations:     node-role.kubernetes.io/master:NoSchedule
node.kubernetes.io/disk-pressure:NoSchedule op=Exists
node.kubernetes.io/memory-pressure:NoSchedule op=Exists
node.kubernetes.io/network-unavailable:NoSchedule op=Exists
node.kubernetes.io/not-ready:NoExecute op=Exists
node.kubernetes.io/pid-pressure:NoSchedule op=Exists
node.kubernetes.io/unreachable:NoExecute op=Exists
node.kubernetes.io/unschedulable:NoSchedule op=Exists
Events:
Type     Reason            Age                   From               Message
----     ------            ----                  ----               -------
Normal   Scheduled         89m                   default-scheduler  Successfully assigned metallb-system/speaker-5kxc9 to taurus
Normal   Pulling           89m                   kubelet            Pulling image "quay.io/metallb/speaker:main"
Normal   Pulled            89m                   kubelet            Successfully pulled image "quay.io/metallb/speaker:main" in 14.647284366s
Normal   Created           89m                   kubelet            Created container speaker
Normal   Started           89m                   kubelet            Started container speaker
Warning  DNSConfigForming  4m14s (x71 over 89m)  kubelet            Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 8.8.8.8 1.1.1.1 192.168.2.1

关于MetalLB控制器的问题。我也面临这个问题,因为我的行动重新安装了金属乐队。参见:https://github.com/metallb/metallb/issues/673

删除

kubectl delete -f https://raw.githubusercontent.com/metallb/metallb/v0.9.5/manifests/metallb.yaml
kubectl delete -f metallb-configMap.yaml

安装
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.9.5/manifests/metallb.yaml
kubectl apply -f metallb-configMap.yaml

不要担心已分配IP的现有服务。当您删除MetalLB控制器时,服务仍然可以正常工作。

我也遵循Kind文档,使用metalb的主版本。我的集群是1.21.1,通过使用特定版本的metallb(即v0.10)使LB工作。

您创建的metalb配置映射似乎有问题。在地址池部分中,您提到分配的IP范围为192.168.2.210 - 192.168.250,这是错误的,因为第二个不是有效的IP (192.168.250不是有效的IP地址)。) .

请使用以下命令编辑configmap,

$ kubectl edit configmap config -n metallb-system

并将范围修正为192.168.2.210 - 192.168.2.250。删除该服务后重新创建。如果它仍然失败,再次张贴到这个线程。希望这对你有帮助:)

我发现我的问题是我选择了一个不在MetalLB的IP范围内的LoadBalancerIP

(即。MetalLB的范围是192.168.100.10 - 192.168.100.20,而我将IP设置为192.168.100.21)

最新更新