Kubernetes中的Nginx控制器:握手到上游-SSL握手中的对等封闭连接



在使用kubeadm构建的本地环境上。集群由主节点和2个工作节点组成。

失败的原因:

通过服务类型LoadBalancer将Nginx入口控制器暴露给外部,尝试将TLS终止到Kubernetes集群。

这里暴露的服务:

NAME            TYPE           CLUSTER-IP      EXTERNAL-IP    PORT(S)                      
nginx-ingress   LoadBalancer   10.101.5.75     192.168.1.82   80:30745/TCP,443:30092/TCP   
web-service     ClusterIP      10.101.26.176   <none>         8080/TCP

它的工作原理:

能够通过HTTP端口80从外部访问web应用程序。

这里是集群中的吊舱:

NAME                              READY   STATUS    RESTARTS   AGE   IP          NODE    
nginx-ingress-7c5588544d-4mw9d    1/1     Running   0          38m   10.44.0.2   node1   
web-deployment-7d84778bc6-52pq7   1/1     Running   1          19h   10.44.0.1   node1   
web-deployment-7d84778bc6-9wwmn   1/1     Running   1          19h   10.36.0.2   node2   

TLS终止的测试结果

客户端:

curl -k https://example.com -v
* Rebuilt URL to https://example.com/
*   Trying 192.168.1.82...
* Connected to example.com (192.168.1.82) port 443 (#0)
* found 127 certificates in /etc/ssl/certs/ca-certificates.crt
* found 513 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
* SSL connection using TLS1.2 / ECDHE_RSA_AES_128_GCM_SHA256
*    server certificate verification SKIPPED
*    server certificate status verification SKIPPED
*    common name: example.com (matched)
*    server certificate expiration date OK
*    server certificate activation date OK
*    certificate public key: RSA
*    certificate version: #1
*    subject: CN=example.com
*    start date: Thu, 22 Oct 2020 16:33:49 GMT
*    expire date: Fri, 22 Oct 2021 16:33:49 GMT
*    issuer: CN=My Cert Authority
*    compression: NULL
* ALPN, server accepted to use http/1.1
> GET / HTTP/1.1
> Host: example.com
> User-Agent: curl/7.47.0
> Accept: */*
> 
< HTTP/1.1 502 Bad Gateway
< Server: nginx/1.19.3

服务器端(来自nginx入口吊舱的日志(:

GET / HTTP/1.1
Connection: close
Host: example.com
X-Real-IP: 10.32.0.1
X-Forwarded-For: 10.32.0.1
X-Forwarded-Host: example.com
X-Forwarded-Port: 443
X-Forwarded-Proto: https
User-Agent: curl/7.47.0
Accept: */*
2020/10/22 18:38:59 [error] 23#23: *12 peer closed connection in SSL handshake while SSL handshaking  to upstream, client: 10.32.0.1, server: example.com, request: "GET / HTTP/1.1", upstream: "https://10.44.0.1:8081/", host: "example.com"
2020/10/22 18:38:59 [warn] 23#23: *12 upstream server temporarily disabled while SSL handshaking to upstream, client: 10.32.0.1, server: example.com, request: "GET / HTTP/1.1", upstream: "https://10.44.0.1:8081/", host: "example.com"
HTTP/1.1 502 Bad Gateway
Server: nginx/1.19.3

我检查了什么

根据链接生成了CA和服务器证书:https://kubernetes.github.io/ingress-nginx/examples/PREREQUISITES/#tls-证书

检查了nginxingress Pod的/etc/nginx/secrets/default下的服务器证书和服务器密钥,它们看起来是正确的。这里的输出资源VirtualServer:

NAME         STATE   HOST          IP             PORTS      AGE
vs-example   Valid   example.com   192.168.1.82   [80,443]   121m

虚拟服务器:

apiVersion: k8s.nginx.org/v1
kind: VirtualServer
metadata:
name: vs-example
namespace: nginx-ingress
spec:
host: example.com
tls:
secret: example-tls
upstreams:
- name: example
service: web-service
port: 8080
tls:
enable: true
routes:
- path: /v2
action:
pass: example
routes:
- path: /
action:
pass: example

秘密

apiVersion: v1
kind: Secret
metadata:
name: example-tls
namespace: nginx-ingress
data:
tls.crt: (..omitted..)
tls.key: (..omitted..)
type: kubernetes.io/tls

Nginx.conf

以下是从运行中的Pod:获取的nginx.conf的摘录

server {
# required to support the Websocket protocol in VirtualServer/VirtualServerRoutes
set $default_connection_header "";
listen 80 default_server;
listen 443 ssl default_server;
ssl_certificate /etc/nginx/secrets/default;
ssl_certificate_key /etc/nginx/secrets/default;
server_name _;
server_tokens "on";

尝试使用HTTPS访问web应用程序时,找不到正在进行的操作。

VirtualServer中的几个错误,用这些更改编辑解决了问题:

apiVersion: k8s.nginx.org/v1
kind: VirtualServer
metadata:
name: vs-example
namespace: nginx-ingress
spec:
host: example.com
tls:
secret: example-tls
upstreams:
- name: example
service: web-service
port: 8080
#    tls:            --> this caused the SSL issue in the upstream
#      enable: true
routes:
- path: /
action:
pass: example
- path: /v1
action:
pass: example

使用HTTPS访问web应用程序现在可以使用

最新更新