我已经使用yaml文件在3个节点的k8s集群(裸机)上安装了kong-ingress-controller(您可以在问题底部看到该文件)，并且一切正常运行:
$kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
default bar-deployment-64d5b5b5c4-99p4q 1/1 Running 0 12m
default foo-deployment-877bf9974-xmpj6 1/1 Running 0 15m
kong ingress-kong-5cd9db4db9-4cg4q 2/2 Running 0 79m
kube-system calico-kube-controllers-5f6cfd688c-5njnn 1/1 Running 0 18h
kube-system calico-node-5k9b6 1/1 Running 0 18h
kube-system calico-node-jbb7k 1/1 Running 0 18h
kube-system calico-node-mmmts 1/1 Running 0 18h
kube-system coredns-74ff55c5b-5q5fn 1/1 Running 0 23h
kube-system coredns-74ff55c5b-9bbbk 1/1 Running 0 23h
kube-system etcd-kubernetes-master 1/1 Running 1 23h
kube-system kube-apiserver-kubernetes-master 1/1 Running 1 23h
kube-system kube-controller-manager-kubernetes-master 1/1 Running 1 23h
kube-system kube-proxy-4h7hs 1/1 Running 0 20h
kube-system kube-proxy-sd6b2 1/1 Running 0 20h
kube-system kube-proxy-v9z8p 1/1 Running 1 23h
kube-system kube-scheduler-kubernetes-master 1/1 Running 1 23h
但是问题在这里:
EXTERNAL_IPofkong-proxy service是所以我不能从外面到达我的集群
$kubectl get services --all-namespaces
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default bar-service ClusterIP 10.103.49.102 <none> 5000/TCP 15m
default foo-service ClusterIP 10.102.52.89 <none> 5000/TCP 19m
default kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 23h
kong kong-proxy LoadBalancer 10.104.79.161 <pending> 80:31583/TCP,443:30053/TCP 82m
kong kong-validation-webhook ClusterIP 10.109.75.104 <none> 443/TCP 82m
kube-system kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 23h
$ kubectl describe service kong-proxy -n kong
Name: kong-proxy
Namespace: kong
Labels: <none>
Annotations: service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp
service.beta.kubernetes.io/aws-load-balancer-type: nlb
Selector: app=ingress-kong
Type: LoadBalancer
IP Families: <none>
IP: 10.104.79.161
IPs: 10.104.79.161
Port: proxy 80/TCP
TargetPort: 8000/TCP
NodePort: proxy 31583/TCP
Endpoints: 192.168.74.69:8000
Port: proxy-ssl 443/TCP
TargetPort: 8443/TCP
NodePort: proxy-ssl 30053/TCP
Endpoints: 192.168.74.69:8443
Session Affinity: None
External Traffic Policy: Cluster
Events: <none>
我的k8s版本是1.20.1和我的docker版本是19.3.10。如果有人能帮我找到一个解决方案，那就太好了
=============================================
kong-ingress-controlleryaml文件:
apiVersion: v1
kind: Namespace
metadata:
name: kong
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: kongclusterplugins.configuration.konghq.com
spec:
additionalPrinterColumns:
- JSONPath: .plugin
description: Name of the plugin
name: Plugin-Type
type: string
- JSONPath: .metadata.creationTimestamp
description: Age
name: Age
type: date
- JSONPath: .disabled
description: Indicates if the plugin is disabled
name: Disabled
priority: 1
type: boolean
- JSONPath: .config
description: Configuration of the plugin
name: Config
priority: 1
type: string
group: configuration.konghq.com
names:
kind: KongClusterPlugin
plural: kongclusterplugins
shortNames:
- kcp
scope: Cluster
subresources:
status: {}
validation:
openAPIV3Schema:
properties:
config:
type: object
configFrom:
properties:
secretKeyRef:
properties:
key:
type: string
name:
type: string
namespace:
type: string
required:
- name
- namespace
- key
type: object
type: object
disabled:
type: boolean
plugin:
type: string
protocols:
items:
enum:
- http
- https
- grpc
- grpcs
- tcp
- tls
type: string
type: array
run_on:
enum:
- first
- second
- all
type: string
required:
- plugin
version: v1
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: kongconsumers.configuration.konghq.com
spec:
additionalPrinterColumns:
- JSONPath: .username
description: Username of a Kong Consumer
name: Username
type: string
- JSONPath: .metadata.creationTimestamp
description: Age
name: Age
type: date
group: configuration.konghq.com
names:
kind: KongConsumer
plural: kongconsumers
shortNames:
- kc
scope: Namespaced
subresources:
status: {}
validation:
openAPIV3Schema:
properties:
credentials:
items:
type: string
type: array
custom_id:
type: string
username:
type: string
version: v1
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: kongingresses.configuration.konghq.com
spec:
group: configuration.konghq.com
names:
kind: KongIngress
plural: kongingresses
shortNames:
- ki
scope: Namespaced
subresources:
status: {}
validation:
openAPIV3Schema:
properties:
proxy:
properties:
connect_timeout:
minimum: 0
type: integer
path:
pattern: ^/.*$
type: string
protocol:
enum:
- http
- https
- grpc
- grpcs
- tcp
- tls
type: string
read_timeout:
minimum: 0
type: integer
retries:
minimum: 0
type: integer
write_timeout:
minimum: 0
type: integer
type: object
route:
properties:
headers:
additionalProperties:
items:
type: string
type: array
type: object
https_redirect_status_code:
type: integer
methods:
items:
type: string
type: array
path_handling:
enum:
- v0
- v1
type: string
preserve_host:
type: boolean
protocols:
items:
enum:
- http
- https
- grpc
- grpcs
- tcp
- tls
type: string
type: array
regex_priority:
type: integer
request_buffering:
type: boolean
response_buffering:
type: boolean
snis:
items:
type: string
type: array
strip_path:
type: boolean
upstream:
properties:
algorithm:
enum:
- round-robin
- consistent-hashing
- least-connections
type: string
hash_fallback:
type: string
hash_fallback_header:
type: string
hash_on:
type: string
hash_on_cookie:
type: string
hash_on_cookie_path:
type: string
hash_on_header:
type: string
healthchecks:
properties:
active:
properties:
concurrency:
minimum: 1
type: integer
healthy:
properties:
http_statuses:
items:
type: integer
type: array
interval:
minimum: 0
type: integer
successes:
minimum: 0
type: integer
type: object
http_path:
pattern: ^/.*$
type: string
timeout:
minimum: 0
type: integer
unhealthy:
properties:
http_failures:
minimum: 0
type: integer
http_statuses:
items:
type: integer
type: array
interval:
minimum: 0
type: integer
tcp_failures:
minimum: 0
type: integer
timeout:
minimum: 0
type: integer
type: object
type: object
passive:
properties:
healthy:
properties:
http_statuses:
items:
type: integer
type: array
interval:
minimum: 0
type: integer
successes:
minimum: 0
type: integer
type: object
unhealthy:
properties:
http_failures:
minimum: 0
type: integer
http_statuses:
items:
type: integer
type: array
interval:
minimum: 0
type: integer
tcp_failures:
minimum: 0
type: integer
timeout:
minimum: 0
type: integer
type: object
type: object
threshold:
type: integer
type: object
host_header:
type: string
slots:
minimum: 10
type: integer
type: object
version: v1
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: kongplugins.configuration.konghq.com
spec:
additionalPrinterColumns:
- JSONPath: .plugin
description: Name of the plugin
name: Plugin-Type
type: string
- JSONPath: .metadata.creationTimestamp
description: Age
name: Age
type: date
- JSONPath: .disabled
description: Indicates if the plugin is disabled
name: Disabled
priority: 1
type: boolean
- JSONPath: .config
description: Configuration of the plugin
name: Config
priority: 1
type: string
group: configuration.konghq.com
names:
kind: KongPlugin
plural: kongplugins
shortNames:
- kp
scope: Namespaced
subresources:
status: {}
validation:
openAPIV3Schema:
properties:
config:
type: object
configFrom:
properties:
secretKeyRef:
properties:
key:
type: string
name:
type: string
required:
- name
- key
type: object
type: object
disabled:
type: boolean
plugin:
type: string
protocols:
items:
enum:
- http
- https
- grpc
- grpcs
- tcp
- tls
type: string
type: array
run_on:
enum:
- first
- second
- all
type: string
required:
- plugin
version: v1
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: tcpingresses.configuration.konghq.com
spec:
additionalPrinterColumns:
- JSONPath: .status.loadBalancer.ingress[*].ip
description: Address of the load balancer
name: Address
type: string
- JSONPath: .metadata.creationTimestamp
description: Age
name: Age
type: date
group: configuration.konghq.com
names:
kind: TCPIngress
plural: tcpingresses
scope: Namespaced
subresources:
status: {}
validation:
openAPIV3Schema:
properties:
apiVersion:
type: string
kind:
type: string
metadata:
type: object
spec:
properties:
rules:
items:
properties:
backend:
properties:
serviceName:
type: string
servicePort:
format: int32
type: integer
type: object
host:
type: string
port:
format: int32
type: integer
type: object
type: array
tls:
items:
properties:
hosts:
items:
type: string
type: array
secretName:
type: string
type: object
type: array
type: object
status:
type: object
version: v1beta1
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: kong-serviceaccount
namespace: kong
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
name: kong-ingress-clusterrole
rules:
- apiGroups:
- ""
resources:
- endpoints
- nodes
- pods
- secrets
verbs:
- list
- watch
- apiGroups:
- ""
resources:
- nodes
verbs:
- get
- apiGroups:
- ""
resources:
- services
verbs:
- get
- list
- watch
- apiGroups:
- networking.k8s.io
- extensions
- networking.internal.knative.dev
resources:
- ingresses
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- events
verbs:
- create
- patch
- apiGroups:
- networking.k8s.io
- extensions
- networking.internal.knative.dev
resources:
- ingresses/status
verbs:
- update
- apiGroups:
- configuration.konghq.com
resources:
- tcpingresses/status
verbs:
- update
- apiGroups:
- configuration.konghq.com
resources:
- kongplugins
- kongclusterplugins
- kongcredentials
- kongconsumers
- kongingresses
- tcpingresses
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- configmaps
verbs:
- create
- get
- update
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: kong-ingress-clusterrole-nisa-binding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: kong-ingress-clusterrole
subjects:
- kind: ServiceAccount
name: kong-serviceaccount
namespace: kong
---
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp
service.beta.kubernetes.io/aws-load-balancer-type: nlb
name: kong-proxy
namespace: kong
spec:
ports:
- name: proxy
port: 80
protocol: TCP
targetPort: 8000
- name: proxy-ssl
port: 443
protocol: TCP
targetPort: 8443
selector:
app: ingress-kong
type: LoadBalancer
---
apiVersion: v1
kind: Service
metadata:
name: kong-validation-webhook
namespace: kong
spec:
ports:
- name: webhook
port: 443
protocol: TCP
targetPort: 8080
selector:
app: ingress-kong
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: ingress-kong
name: ingress-kong
namespace: kong
spec:
replicas: 1
selector:
matchLabels:
app: ingress-kong
template:
metadata:
annotations:
kuma.io/gateway: enabled
prometheus.io/port: "8100"
prometheus.io/scrape: "true"
traffic.sidecar.istio.io/includeInboundPorts: ""
labels:
app: ingress-kong
spec:
containers:
- env:
- name: KONG_PROXY_LISTEN
value: 0.0.0.0:8000, 0.0.0.0:8443 ssl http2
- name: KONG_PORT_MAPS
value: 80:8000, 443:8443
- name: KONG_ADMIN_LISTEN
value: 127.0.0.1:8444 ssl
- name: KONG_STATUS_LISTEN
value: 0.0.0.0:8100
- name: KONG_DATABASE
value: "off"
- name: KONG_NGINX_WORKER_PROCESSES
value: "2"
- name: KONG_ADMIN_ACCESS_LOG
value: /dev/stdout
- name: KONG_ADMIN_ERROR_LOG
value: /dev/stderr
- name: KONG_PROXY_ERROR_LOG
value: /dev/stderr
image: kong:2.5
lifecycle:
preStop:
exec:
command:
- /bin/sh
- -c
- kong quit
livenessProbe:
failureThreshold: 3
httpGet:
path: /status
port: 8100
scheme: HTTP
initialDelaySeconds: 5
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
name: proxy
ports:
- containerPort: 8000
name: proxy
protocol: TCP
- containerPort: 8443
name: proxy-ssl
protocol: TCP
- containerPort: 8100
name: metrics
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /status
port: 8100
scheme: HTTP
initialDelaySeconds: 5
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
- env:
- name: CONTROLLER_KONG_ADMIN_URL
value: https://127.0.0.1:8444
- name: CONTROLLER_KONG_ADMIN_TLS_SKIP_VERIFY
value: "true"
- name: CONTROLLER_PUBLISH_SERVICE
value: kong/kong-proxy
- name: POD_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
image: kong/kubernetes-ingress-controller:1.3
imagePullPolicy: IfNotPresent
livenessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 5
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
name: ingress-controller
ports:
- containerPort: 8080
name: webhook
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 5
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
serviceAccountName: kong-serviceaccount
简短的回答是@iglen_在这个回答中所说的，但我决定解释一下解决方案。
当使用云提供商时，服务的LoadBalancer类型将由环境自动管理和供应(参见k8s文档)，但是当创建您自己的裸机集群时，您将需要添加服务，该服务将管理LoadBalancer类型服务的IPs供应。Metal-LB就是这样一个服务，它就是为此而构建的。
安装前MetalLB检查要求
在部署MetalLB之前，我们需要做一个步骤:
如果您在IPVS模式下使用kube-proxy，从Kubernetes v1.14.2开始必须启用严格ARP模式。
注意，如果你使用kube-router作为服务代理，你不需要这个，因为默认情况下它是启用严格ARP的。输入命令:
$ kubectl edit configmap -n kube-system kube-proxy
在打开的页面中搜索模式在我的情况下，模式等于空字符串，所以我不需要改变任何东西，但在这种情况下，模式被设置为ipvs，正如它在安装指南中所说，你需要在这个文件中设置以下配置:
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
mode: "ipvs"
ipvs:
strictARP: true
作为下一步您需要运行的命令:
$ kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.10.2/manifests/namespace.yaml
$ kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.10.2/manifests/metallb.yaml
$ kubectl create secret generic -n metallb-system memberlist --from-literal=secretkey="$(openssl rand -base64 128)"
在运行以上命令后，我们得到了这个:
$ kubectl get all -n metallb-system
NAME READY STATUS RESTARTS AGE
pod/controller-6b78sff7d9-2rv2f 1/1 Running 0 3m
pod/speaker-7bqev 1/1 Running 0 3m
pod/speaker-txrg5 1/1 Running 0 3m
pod/speaker-w7th5 1/1 Running 0 3m
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
daemonset.apps/speaker 3 3 3 3 3 kubernetes.io/os=linux 3m
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/controller 1/1 1 1 3m
NAME DESIRED CURRENT READY AGE
replicaset.apps/controller-6b78sff7d9 1 1 1 3m
MetalLB需要一些IPv4 addresses:
$ ip a s
1: ens160: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
[...]
inet 10.240.1.59/24 brd 10.240.1.255 scope global dynamic noprefixroute ens160
valid_lft 425669sec preferred_lft 421269sec
[...]
ens160是我的控制平面网络接口，正如你所看到的，它的ip范围是10.240.1.59/24，所以我要在这个网络中分配一组ip地址:
$ sipcalc 10.240.1.59/24
-[ipv4 : 10.240.1.59/24] - 0
[CIDR]
Host address - 10.240.1.59
Host address (decimal) - 183500115
Host address (hex) - AF0031B
Network address - 10.240.1.0
Network mask - 255.255.255.0
Network mask (bits) - 24
Network mask (hex) - FFFFF000
Broadcast address - 10.240.1.255
Cisco wildcard - 0.0.0.255
Addresses in network - 256
Network range - 10.240.1.0 - 10.240.1.255
Usable range - 10.240.1.1 - 10.240.1.254
现在我要取10从Usable range中获取ip地址并将其分配给MetalLB。让我们为MetalLB创建一个configmap:
$ sudo nano metallb-cm.yaml
将下面的配置粘贴到metallb-cm.yaml:
apiVersion: v1
kind: ConfigMap
metadata:
namespace: metallb-system
name: config
data:
config: |
address-pools:
- name: default
protocol: layer2
addresses:
- 10.240.1.100-10.240.1.110
然后保存文件并运行以下命令:
$ kubectl create -f metallb-cm.yaml
现在让我们再次检查我们的服务:
$ kubectl get services --all-namespaces
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default bar-service ClusterIP 10.103.49.102 <none> 5000/TCP 15m
default foo-service ClusterIP 10.102.52.89 <none> 5000/TCP 19m
default kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 23h
kong kong-proxy LoadBalancer 10.104.79.161 10.240.1.100 80:31583/TCP,443:30053/TCP 82m
kong kong-validation-webhook ClusterIP 10.109.75.104 <none> 443/TCP 82m
kube-system kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 23h
您可以看到类型为LoadBalancer的服务现在有一个ip地址。
有同样的问题，经过几天的寻找解决方案，我遇到了metallb，从nginx入口安装在裸机上
MetalLB为Kubernetes提供了一个网络负载平衡器实现不能在受支持的云提供商上有效运行的集群允许在任何集群中使用LoadBalancer服务
，从他们的文档中我得到了这个
Kubernetes没有为裸机集群提供网络负载均衡器(LoadBalancer类型的服务)的实现。Kubernetes所提供的网络负载平衡器的实现都是调用各种IaaS平台(GCP、AWS、Azure…)的粘合代码。如果你不是在支持的IaaS平台上运行(GCP, AWS, Azure…)，loadbalancer在创建时将无限期地保持在"pending"状态。
我没有完成安装，但我希望上面的解释能回答你关于外部ip等待状态的问题