TCP入口点的Nginx入口不工作

Question

我使用Nginx作为Kubernetes Ingress控制器。在遵循这个简单的例子之后，我能够设置这个例子

现在我正试图用以下配置为logstash设置TCP入口点

Logstash：

apiVersion: v1
kind: Secret
metadata:
name: logstash-secret
namespace: kube-logging
type: Opaque
data:
tls.crt: "<base64 encoded>" #For logstash.test.domain.com
tls.key: "<base64 encoded>" #For logstash.test.domain.com
---
apiVersion: v1
kind: ConfigMap
metadata:
name: logstash-config
namespace: kube-logging
labels:
app: logstash
data:
syslog.conf: |-
input {
tcp {
port => 5050
type => syslog
}
}
filter {
grok {
match => {"message" => "%{SYSLOGLINE}"}
}
}
output {
elasticsearch {
hosts => ["http://elasticsearch:9200"] #elasticsearch running in same namespace (kube-logging)
index => "syslog-%{+YYYY.MM.dd}"
}
stdout { codec => rubydebug }
}
---
kind: Deployment
apiVersion: apps/v1
metadata:
name: logstash
namespace: kube-logging
labels:
app: logstash
spec:
replicas: 1
selector:
matchLabels:
app: logstash
strategy:
type: RollingUpdate
template:
metadata:
labels:
app: logstash
spec:
#serviceAccountName: logstash
containers:
- name: logstash
image: docker.elastic.co/logstash/logstash:7.2.1
imagePullPolicy: Always
env:
- name: ELASTICSEARCH_HOST
value: elasticsearch
- name: ELASTICSEARCH_PORT
value: "9200"
- name: ELASTICSEARCH_USERNAME
value: elastic
- name: ELASTICSEARCH_PASSWORD
value: changeme
- name: ELASTIC_CLOUD_ID
value:
- name: ELASTIC_CLOUD_AUTH
value:
ports:
- name: logstash
containerPort: 5050
protocol: TCP
securityContext:
runAsUser: 0
volumeMounts:
- name: config
mountPath: /usr/share/logstash/pipeline/syslog.conf
readOnly: true
subPath: syslog.conf
volumes:
- name: config
configMap:
defaultMode: 0600
name: logstash-config
---
kind: Service
apiVersion: v1
metadata:
name: logstash
namespace: kube-logging
labels:
app: logstash
spec:
ports:
- name: tcp-port
protocol: TCP
port: 5050
targetPort: 5050
selector:
app: logstash

Nginx入口：

apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-ingress
namespace: kube-ingress
spec:
replicas: 1
selector:
matchLabels:
app: nginx-ingress
template:
metadata:
labels:
app: nginx-ingress
spec:
serviceAccountName: nginx-ingress
containers:
- image: nginx/nginx-ingress:1.5.7
imagePullPolicy: Always
name: nginx-ingress
ports:
- name: http
containerPort: 80
- name: https
containerPort: 443
- name: tcp5050
containerPort: 5050
securityContext:
allowPrivilegeEscalation: true
runAsUser: 101 #nginx
capabilities:
drop:
- ALL
add:
- NET_BIND_SERVICE
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
args:
- -nginx-configmaps=$(POD_NAMESPACE)/nginx-config
- -default-server-tls-secret=$(POD_NAMESPACE)/default-server-secret
- -v=3 # Enables extensive logging. Useful for troubleshooting.
#- -report-ingress-status
#- -external-service=nginx-ingress
#- -enable-leader-election
#- -enable-prometheus-metrics
#- -enable-custom-resources

负载均衡器：

apiVersion: v1
kind: Service
metadata:
name: nginx-ingress-external
namespace: kube-ingress
annotations:
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "tcp"
service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: "*"
service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
spec:
type: LoadBalancer
ports:
- name: http
protocol: TCP
port: 80
targetPort: 80
- name: https
protocol: TCP
port: 443
targetPort: 443
- name: tcp5050
protocol: TCP
port: 5050
targetPort: 5050
selector:
app: nginx-ingress

入口：

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: logstash-ingress
namespace: kube-logging
spec:
tls:
- hosts:
- logstash.test.domain.com
secretName: logstash-secret #This has self-signed cert for logstash.test.domain.com
rules:
- host: logstash.test.domain.com
http:
paths:
- path: /
backend:
serviceName: logstash
servicePort: 5050

有了这个配置，它显示如下，

NAME               HOSTS                         ADDRESS   PORTS     AGE
logstash-ingress   logstash.test.domain.com                 80, 443   79m

为什么此处未列出端口5050？

只想通过公共端点公开logstash服务。当我使用集群内的openssl s_client -connect logstash.kube-logging.svc.cluster.local:5050我得到

$ openssl s_client -connect logstash.kube-logging.svc.cluster.local:5050
CONNECTED(00000005)

但从集群openssl s_client -connect logstash.test.domain.com:5050之外，我得到了

$ openssl s_client -connect logstash.test.domain.com:5050
connect: Connection refused
connect:errno=61

和

$ openssl s_client -cert logstash_test_domain_com.crt -key logstash_test_domain_com.key -servername logstash.test.domain.com:5050
connect: Connection refused
connect:errno=61

我需要做些什么才能让它发挥作用？

Answer 1

你似乎有点困惑。因此，让我们从订购您的服务和入口开始。

首先，kubernetes中有3种类型的服务。ClusterIP，它允许您在k8s中公开内部部署。Nodeport与ClusterIP相同，但也通过每个节点的外部IP和端口公开您的部署，端口在30K-32K范围内。最后是LoadBalancer服务，它与ClusterIP相同，但也在云提供商LoadBalancer指定的特定外部IP地址中公开您的应用程序。

您创建的NodePort服务将使logstash可以通过30K到32K范围内的随机端口中的每个节点外部IP访问；找到运行kubectl get services | grep nginx-ingress的端口并检查最后一列。要获取节点的外部ip地址，请运行kubectl get node -o wide。您创建的LoadBalancer服务将使logstash可以通过端口5050中的外部IP地址访问。要查找IP，请运行kubectl get services | grep nginx-ingress-external。最后，您还创建了一个入口资源来访问logstash。为此，您定义了一个主机，该主机在端口443中可以访问给定的TLS，并且入站流量将重定向到端口5050中logstash的服务类型ClusterIP。所以，你有三种方法可以到达logstash。我会选择LoadBalancer，因为它是一个特定的端口。