在使用kubeadm构建的本地环境上。集群由主节点和2个工作节点组成。
失败的原因:
通过服务类型LoadBalancer将Nginx入口控制器暴露给外部,尝试将TLS终止到Kubernetes集群。
这里暴露的服务:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S)
nginx-ingress LoadBalancer 10.101.5.75 192.168.1.82 80:30745/TCP,443:30092/TCP
web-service ClusterIP 10.101.26.176 <none> 8080/TCP
它的工作原理:
能够通过HTTP端口80从外部访问web应用程序。
这里是集群中的吊舱:
NAME READY STATUS RESTARTS AGE IP NODE
nginx-ingress-7c5588544d-4mw9d 1/1 Running 0 38m 10.44.0.2 node1
web-deployment-7d84778bc6-52pq7 1/1 Running 1 19h 10.44.0.1 node1
web-deployment-7d84778bc6-9wwmn 1/1 Running 1 19h 10.36.0.2 node2
TLS终止的测试结果
客户端:
curl -k https://example.com -v
* Rebuilt URL to https://example.com/
* Trying 192.168.1.82...
* Connected to example.com (192.168.1.82) port 443 (#0)
* found 127 certificates in /etc/ssl/certs/ca-certificates.crt
* found 513 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
* SSL connection using TLS1.2 / ECDHE_RSA_AES_128_GCM_SHA256
* server certificate verification SKIPPED
* server certificate status verification SKIPPED
* common name: example.com (matched)
* server certificate expiration date OK
* server certificate activation date OK
* certificate public key: RSA
* certificate version: #1
* subject: CN=example.com
* start date: Thu, 22 Oct 2020 16:33:49 GMT
* expire date: Fri, 22 Oct 2021 16:33:49 GMT
* issuer: CN=My Cert Authority
* compression: NULL
* ALPN, server accepted to use http/1.1
> GET / HTTP/1.1
> Host: example.com
> User-Agent: curl/7.47.0
> Accept: */*
>
< HTTP/1.1 502 Bad Gateway
< Server: nginx/1.19.3
服务器端(来自nginx入口吊舱的日志(:
GET / HTTP/1.1
Connection: close
Host: example.com
X-Real-IP: 10.32.0.1
X-Forwarded-For: 10.32.0.1
X-Forwarded-Host: example.com
X-Forwarded-Port: 443
X-Forwarded-Proto: https
User-Agent: curl/7.47.0
Accept: */*
2020/10/22 18:38:59 [error] 23#23: *12 peer closed connection in SSL handshake while SSL handshaking to upstream, client: 10.32.0.1, server: example.com, request: "GET / HTTP/1.1", upstream: "https://10.44.0.1:8081/", host: "example.com"
2020/10/22 18:38:59 [warn] 23#23: *12 upstream server temporarily disabled while SSL handshaking to upstream, client: 10.32.0.1, server: example.com, request: "GET / HTTP/1.1", upstream: "https://10.44.0.1:8081/", host: "example.com"
HTTP/1.1 502 Bad Gateway
Server: nginx/1.19.3
我检查了什么
根据链接生成了CA和服务器证书:https://kubernetes.github.io/ingress-nginx/examples/PREREQUISITES/#tls-证书
检查了nginxingress Pod的/etc/nginx/secrets/default下的服务器证书和服务器密钥,它们看起来是正确的。这里的输出资源VirtualServer:
NAME STATE HOST IP PORTS AGE
vs-example Valid example.com 192.168.1.82 [80,443] 121m
虚拟服务器:
apiVersion: k8s.nginx.org/v1
kind: VirtualServer
metadata:
name: vs-example
namespace: nginx-ingress
spec:
host: example.com
tls:
secret: example-tls
upstreams:
- name: example
service: web-service
port: 8080
tls:
enable: true
routes:
- path: /v2
action:
pass: example
routes:
- path: /
action:
pass: example
秘密
apiVersion: v1
kind: Secret
metadata:
name: example-tls
namespace: nginx-ingress
data:
tls.crt: (..omitted..)
tls.key: (..omitted..)
type: kubernetes.io/tls
Nginx.conf
以下是从运行中的Pod:获取的nginx.conf的摘录
server {
# required to support the Websocket protocol in VirtualServer/VirtualServerRoutes
set $default_connection_header "";
listen 80 default_server;
listen 443 ssl default_server;
ssl_certificate /etc/nginx/secrets/default;
ssl_certificate_key /etc/nginx/secrets/default;
server_name _;
server_tokens "on";
尝试使用HTTPS访问web应用程序时,找不到正在进行的操作。
VirtualServer中的几个错误,用这些更改编辑解决了问题:
apiVersion: k8s.nginx.org/v1
kind: VirtualServer
metadata:
name: vs-example
namespace: nginx-ingress
spec:
host: example.com
tls:
secret: example-tls
upstreams:
- name: example
service: web-service
port: 8080
# tls: --> this caused the SSL issue in the upstream
# enable: true
routes:
- path: /
action:
pass: example
- path: /v1
action:
pass: example
使用HTTPS访问web应用程序现在可以使用