为什么LoadBalancer服务仍处于挂起状态



我在AWS上使用KubeSpray创建了我的kubernetes集群。现在我正试图让入口控制器工作。我的理解是我需要申请https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.34.1/deploy/static/provider/aws/deploy.yaml这将创建我需要的所有资源,包括网络负载均衡器。

然而,LoadBalancer从未脱离挂起状态:

$ kubectl -n ingress-nginx get svc
NAME                                 TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx-controller             LoadBalancer   10.233.28.147   <pending>     80:31304/TCP,443:31989/TCP   11m
ingress-nginx-controller-admission   ClusterIP      10.233.58.231   <none>        443/TCP                      11m

描述这项服务似乎并没有提供任何有趣的信息。

$ kubectl -n ingress-nginx describe service ingress-nginx-controller
Name:                     ingress-nginx-controller
Namespace:                ingress-nginx
Labels:                   app.kubernetes.io/component=controller
app.kubernetes.io/instance=ingress-nginx
app.kubernetes.io/managed-by=Helm
app.kubernetes.io/name=ingress-nginx
app.kubernetes.io/version=0.34.1
helm.sh/chart=ingress-nginx-2.11.1
Annotations:              kubectl.kubernetes.io/last-applied-configuration:
{"apiVersion":"v1","kind":"Service","metadata":{"annotations":{"service.beta.kubernetes.io/aws-load-balancer-backend-protocol":"tcp","serv...
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp
service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: 60
service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: true
service.beta.kubernetes.io/aws-load-balancer-type: nlb
Selector:                 app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginx
Type:                     LoadBalancer
IP:                       10.233.28.147
Port:                     http  80/TCP
TargetPort:               http/TCP
NodePort:                 http  31304/TCP
Endpoints:                10.233.97.22:80
Port:                     https  443/TCP
TargetPort:               https/TCP
NodePort:                 https  31989/TCP
Endpoints:                10.233.97.22:443
Session Affinity:         None
External Traffic Policy:  Local
HealthCheck NodePort:     30660
Events:                   <none>

如何调试此问题?

更新:

kubectl -n kube-system logs -l component=kube-controller-manager的输出为:

E0801 21:12:29.429759       1 job_controller.go:793] pods "ingress-nginx-admission-create-" is forbidden: error looking up service account ingress-nginx/ingress-nginx-admission: serviceaccount "ingress-nginx-admission" not found
E0801 21:12:29.429788       1 job_controller.go:398] Error syncing job: pods "ingress-nginx-admission-create-" is forbidden: error looking up service account ingress-nginx/ingress-nginx-admission: serviceaccount "ingress-nginx-admission" not found
I0801 21:12:29.429851       1 event.go:278] Event(v1.ObjectReference{Kind:"Job", Namespace:"ingress-nginx", Name:"ingress-nginx-admission-create", UID:"4faad8c5-9b1e-4c23-a942-94be181d590f", APIVersion:"batch/v1", ResourceVersion:"1506255", FieldPath:""}): type: 'Warning' reason: 'FailedCreate' Error creating: pods "ingress-nginx-admission-create-" is forbidden: error looking up service account ingress-nginx/ingress-nginx-admission: serviceaccount "ingress-nginx-admission" not found
E0801 21:12:29.483485       1 job_controller.go:793] pods "ingress-nginx-admission-patch-" is forbidden: error looking up service account ingress-nginx/ingress-nginx-admission: serviceaccount "ingress-nginx-admission" not found
E0801 21:12:29.483512       1 job_controller.go:398] Error syncing job: pods "ingress-nginx-admission-patch-" is forbidden: error looking up service account ingress-nginx/ingress-nginx-admission: serviceaccount "ingress-nginx-admission" not found
I0801 21:12:29.483679       1 event.go:278] Event(v1.ObjectReference{Kind:"Job", Namespace:"ingress-nginx", Name:"ingress-nginx-admission-patch", UID:"92ee0e43-2711-4b37-9fd6-958ef3c95b31", APIVersion:"batch/v1", ResourceVersion:"1506257", FieldPath:""}): type: 'Warning' reason: 'FailedCreate' Error creating: pods "ingress-nginx-admission-patch-" is forbidden: error looking up service account ingress-nginx/ingress-nginx-admission: serviceaccount "ingress-nginx-admission" not found
I0801 21:12:39.436590       1 event.go:278] Event(v1.ObjectReference{Kind:"Job", Namespace:"ingress-nginx", Name:"ingress-nginx-admission-create", UID:"4faad8c5-9b1e-4c23-a942-94be181d590f", APIVersion:"batch/v1", ResourceVersion:"1506255", FieldPath:""}): type: 'Normal' reason: 'SuccessfulCreate' Created pod: ingress-nginx-admission-create-85x58
I0801 21:12:39.489303       1 event.go:278] Event(v1.ObjectReference{Kind:"Job", Namespace:"ingress-nginx", Name:"ingress-nginx-admission-patch", UID:"92ee0e43-2711-4b37-9fd6-958ef3c95b31", APIVersion:"batch/v1", ResourceVersion:"1506257", FieldPath:""}): type: 'Normal' reason: 'SuccessfulCreate' Created pod: ingress-nginx-admission-patch-sn8xv
I0801 21:12:41.448425       1 event.go:278] Event(v1.ObjectReference{Kind:"Job", Namespace:"ingress-nginx", Name:"ingress-nginx-admission-create", UID:"4faad8c5-9b1e-4c23-a942-94be181d590f", APIVersion:"batch/v1", ResourceVersion:"1506297", FieldPath:""}): type: 'Normal' reason: 'Completed' Job completed
I0801 21:12:42.481264       1 event.go:278] Event(v1.ObjectReference{Kind:"Job", Namespace:"ingress-nginx", Name:"ingress-nginx-admission-patch", UID:"92ee0e43-2711-4b37-9fd6-958ef3c95b31", APIVersion:"batch/v1", ResourceVersion:"1506304", FieldPath:""}): type: 'Normal' reason: 'Completed' Job completed

我确实启用了PodSecurityPolicy准入控制器。我使用以下更改更新了deploy.yaml文件。

  • 将以下内容添加到所有ClusterRole和Role资源中
- apiGroups:      [policy]
resources:      [podsecuritypolicies]
resourceNames:  [privileged]
verbs:          [use]
  • 在文件末尾添加以下内容
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
labels:
helm.sh/chart: ingress-nginx-2.11.1
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/version: 0.34.1
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/component: controller
name: ingress-nginx
namespace: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: ingress-nginx
subjects:
- kind: ServiceAccount
name: ingress-nginx
namespace: default

问题回答:

  • IAM角色由Kubesraycontrib/terraform/aws目录中的ansible剧本创建。

  • 那些易解析的脚本为apiserver创建了一个经典的负载均衡器。

这个问题我有两个答案。

one-将cloud-provider选项添加到ansible-playbook命令中,如下所示。

ansible-playbook 
-vvvvv 
-i ./inventory/hosts 
./cluster.yml 
-e ansible_user=centos 
-e cloud_provider=aws 
-e bootstrap_os=centos 
--become 
--become-user=root 
--flush-cache 
-e ansible_ssh_private_key_file=$PKI_PRIVATE_PEM 
| tee kubespray-cluster-$(date "+%Y-%m-%d_%H:%M").log

两个

取消注释group_vars/all.yml中的cloud_provider选项,并将其设置为"aws">

证明

我试过了第一个答案。

$ kubectl -n ingress-nginx get svc
NAME                                 TYPE           CLUSTER-IP      EXTERNAL-IP                                                                     PORT(S)                      AGE
ingress-nginx-controller             LoadBalancer   10.233.57.196   aa....amazonaws.com   80:32111/TCP,443:31854/TCP   109s
ingress-nginx-controller-admission   ClusterIP      10.233.11.133   <none>                                                                          443/TCP                      109s

最新更新