我在AWS上使用KubeSpray创建了我的kubernetes集群。现在我正试图让入口控制器工作。我的理解是我需要申请https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.34.1/deploy/static/provider/aws/deploy.yaml这将创建我需要的所有资源,包括网络负载均衡器。
然而,LoadBalancer从未脱离挂起状态:
$ kubectl -n ingress-nginx get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-controller LoadBalancer 10.233.28.147 <pending> 80:31304/TCP,443:31989/TCP 11m
ingress-nginx-controller-admission ClusterIP 10.233.58.231 <none> 443/TCP 11m
描述这项服务似乎并没有提供任何有趣的信息。
$ kubectl -n ingress-nginx describe service ingress-nginx-controller
Name: ingress-nginx-controller
Namespace: ingress-nginx
Labels: app.kubernetes.io/component=controller
app.kubernetes.io/instance=ingress-nginx
app.kubernetes.io/managed-by=Helm
app.kubernetes.io/name=ingress-nginx
app.kubernetes.io/version=0.34.1
helm.sh/chart=ingress-nginx-2.11.1
Annotations: kubectl.kubernetes.io/last-applied-configuration:
{"apiVersion":"v1","kind":"Service","metadata":{"annotations":{"service.beta.kubernetes.io/aws-load-balancer-backend-protocol":"tcp","serv...
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp
service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: 60
service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: true
service.beta.kubernetes.io/aws-load-balancer-type: nlb
Selector: app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginx
Type: LoadBalancer
IP: 10.233.28.147
Port: http 80/TCP
TargetPort: http/TCP
NodePort: http 31304/TCP
Endpoints: 10.233.97.22:80
Port: https 443/TCP
TargetPort: https/TCP
NodePort: https 31989/TCP
Endpoints: 10.233.97.22:443
Session Affinity: None
External Traffic Policy: Local
HealthCheck NodePort: 30660
Events: <none>
如何调试此问题?
更新:
kubectl -n kube-system logs -l component=kube-controller-manager
的输出为:
E0801 21:12:29.429759 1 job_controller.go:793] pods "ingress-nginx-admission-create-" is forbidden: error looking up service account ingress-nginx/ingress-nginx-admission: serviceaccount "ingress-nginx-admission" not found
E0801 21:12:29.429788 1 job_controller.go:398] Error syncing job: pods "ingress-nginx-admission-create-" is forbidden: error looking up service account ingress-nginx/ingress-nginx-admission: serviceaccount "ingress-nginx-admission" not found
I0801 21:12:29.429851 1 event.go:278] Event(v1.ObjectReference{Kind:"Job", Namespace:"ingress-nginx", Name:"ingress-nginx-admission-create", UID:"4faad8c5-9b1e-4c23-a942-94be181d590f", APIVersion:"batch/v1", ResourceVersion:"1506255", FieldPath:""}): type: 'Warning' reason: 'FailedCreate' Error creating: pods "ingress-nginx-admission-create-" is forbidden: error looking up service account ingress-nginx/ingress-nginx-admission: serviceaccount "ingress-nginx-admission" not found
E0801 21:12:29.483485 1 job_controller.go:793] pods "ingress-nginx-admission-patch-" is forbidden: error looking up service account ingress-nginx/ingress-nginx-admission: serviceaccount "ingress-nginx-admission" not found
E0801 21:12:29.483512 1 job_controller.go:398] Error syncing job: pods "ingress-nginx-admission-patch-" is forbidden: error looking up service account ingress-nginx/ingress-nginx-admission: serviceaccount "ingress-nginx-admission" not found
I0801 21:12:29.483679 1 event.go:278] Event(v1.ObjectReference{Kind:"Job", Namespace:"ingress-nginx", Name:"ingress-nginx-admission-patch", UID:"92ee0e43-2711-4b37-9fd6-958ef3c95b31", APIVersion:"batch/v1", ResourceVersion:"1506257", FieldPath:""}): type: 'Warning' reason: 'FailedCreate' Error creating: pods "ingress-nginx-admission-patch-" is forbidden: error looking up service account ingress-nginx/ingress-nginx-admission: serviceaccount "ingress-nginx-admission" not found
I0801 21:12:39.436590 1 event.go:278] Event(v1.ObjectReference{Kind:"Job", Namespace:"ingress-nginx", Name:"ingress-nginx-admission-create", UID:"4faad8c5-9b1e-4c23-a942-94be181d590f", APIVersion:"batch/v1", ResourceVersion:"1506255", FieldPath:""}): type: 'Normal' reason: 'SuccessfulCreate' Created pod: ingress-nginx-admission-create-85x58
I0801 21:12:39.489303 1 event.go:278] Event(v1.ObjectReference{Kind:"Job", Namespace:"ingress-nginx", Name:"ingress-nginx-admission-patch", UID:"92ee0e43-2711-4b37-9fd6-958ef3c95b31", APIVersion:"batch/v1", ResourceVersion:"1506257", FieldPath:""}): type: 'Normal' reason: 'SuccessfulCreate' Created pod: ingress-nginx-admission-patch-sn8xv
I0801 21:12:41.448425 1 event.go:278] Event(v1.ObjectReference{Kind:"Job", Namespace:"ingress-nginx", Name:"ingress-nginx-admission-create", UID:"4faad8c5-9b1e-4c23-a942-94be181d590f", APIVersion:"batch/v1", ResourceVersion:"1506297", FieldPath:""}): type: 'Normal' reason: 'Completed' Job completed
I0801 21:12:42.481264 1 event.go:278] Event(v1.ObjectReference{Kind:"Job", Namespace:"ingress-nginx", Name:"ingress-nginx-admission-patch", UID:"92ee0e43-2711-4b37-9fd6-958ef3c95b31", APIVersion:"batch/v1", ResourceVersion:"1506304", FieldPath:""}): type: 'Normal' reason: 'Completed' Job completed
我确实启用了PodSecurityPolicy准入控制器。我使用以下更改更新了deploy.yaml
文件。
- 将以下内容添加到所有ClusterRole和Role资源中
- apiGroups: [policy]
resources: [podsecuritypolicies]
resourceNames: [privileged]
verbs: [use]
- 在文件末尾添加以下内容
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
labels:
helm.sh/chart: ingress-nginx-2.11.1
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/version: 0.34.1
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/component: controller
name: ingress-nginx
namespace: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: ingress-nginx
subjects:
- kind: ServiceAccount
name: ingress-nginx
namespace: default
问题回答:
IAM角色由Kubesray
contrib/terraform/aws
目录中的ansible剧本创建。那些易解析的脚本为apiserver创建了一个经典的负载均衡器。
one-将cloud-provider
选项添加到ansible-playbook
命令中,如下所示。
ansible-playbook
-vvvvv
-i ./inventory/hosts
./cluster.yml
-e ansible_user=centos
-e cloud_provider=aws
-e bootstrap_os=centos
--become
--become-user=root
--flush-cache
-e ansible_ssh_private_key_file=$PKI_PRIVATE_PEM
| tee kubespray-cluster-$(date "+%Y-%m-%d_%H:%M").log
两个
取消注释group_vars/all.yml中的cloud_provider选项,并将其设置为"aws">
证明
我试过了第一个答案。
$ kubectl -n ingress-nginx get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-controller LoadBalancer 10.233.57.196 aa....amazonaws.com 80:32111/TCP,443:31854/TCP 109s
ingress-nginx-controller-admission ClusterIP 10.233.11.133 <none> 443/TCP 109s