小贝子编程

Istio | Envoy Proxy Problem: 0 NR filter_chain_not_found | T



我有一个小问题与Istio和EnvoyProxy:NR filter_chain_not_found

套接字客户端和套接字服务器运行在同一个集群(分离的docker-container)中，并间隔发送明文消息。套接字服务器运行在端口50000上，套接字客户端运行在端口50001上。没有mTLS(允许)，通信工作没有问题。如果我激活mTLS (STRICT)，就会出现下面列出的错误。我已经试过写EnvoyFilters了，但是我不能想象这是正确的方法。

来自envoy-proxy的日志:

在Socket服务器端:

[2023-01-16T19:52:55.941Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 5000 - "-" "-" "-" "-" "-" - - 10.1.2.142:50000 10.1.2.146:50001 - -

[2023-01-16T19:58:05.909Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 5001 - "-" "-" "-" "-" "-" - - 10.1.2.142:50000 10.1.2.146:50001 - -

在Socket客户端: 

Connect to SocketServer...  server-c-socket-server-service.server-c-socket-server.svc.cluster.local
SERVER_NAME as string => server-c-socket-server-service.server-c-socket-server.svc.cluster.local
Traceback (most recent call last):
File "/service/server-c-socket-client.py", line 94, in <module>
main()
File "/service/server-c-socket-client.py", line 91, in main
ConnectToSocketServer(SERVER_NAME)
File "/service/server-c-socket-client.py", line 60, in ConnectToSocketServer
answer = con.recv(1024)
^^^^^^^^^^^^^^
ConnectionResetError: [Errno 104] Connection reset by peer

更多信息:

istio-strict-meshpolicy.yaml

apiVersion: security.istio.io/v1beta1 
kind: PeerAuthentication
metadata:
name: "default"
namespace: "istio-system"
spec:
mtls:
mode: STRICT

istio-virtualservice-socket-client.yaml

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: server-c-socket-client-virtualservice
namespace: server-c-socket-client
spec:
hosts:
- server-c-socket-client-service.server-c-socket-client.svc.cluster.local
tcp:
- match:
- port: 50001
route:
- destination:
host: server-c-socket-client-service.server-c-socket-client.svc.cluster.local
port:
number: 50001
weight: 100

istio-virtualservice-socket-server.yaml

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: server-c-socket-server-virtualservice
namespace: server-c-socket-server
spec:
hosts: server-c-socket-server-service.server-c-socket-server.svc.cluster.local
tcp:
route:
- destination:
host: server-c-socket-server-service.server-c-socket-server.svc.cluster.local
port:
number: 50000
weight: 100

istio-destinationrule-socket-client.yaml

apiVersion: networking.istio.io/v1alpha3 
kind: DestinationRule 
metadata:
name: server-c-socket-client-destinationrule
namespace: server-c-socket-client 
spec:
host: server-c-socket-client-service.server-c-socket-client.svc.cluster.local
trafficPolicy:
tls:
mode: MUTUAL
credentialName: cacerts
sni: server-c-socket-client-service.server-c-socket-client.svc.cluster.local

istio-destinationrule-socket-server.yaml

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule 
metadata:
name: server-c-socket-server-destinationrule
namespace: server-c-socket-server
spec:
host: server-c-socket-server-service.server-c-socket-server.svc.cluster.local
trafficPolicy:
tls:
mode: MUTUAL
credentialName: cacerts
sni: server-c-socket-server-service.server-c-socket-server.svc.cluster.local

istio-peerauthentication-socket-client.yaml

apiVersion: security.istio.io/v1beta1 
kind: PeerAuthentication 
metadata:
name: server-c-socket-client-peerauthentication
namespace: server-c-socket-client
spec:
mtls:
mode: STRICT

istio-peerauthentication-socket-server.yaml

apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
name: server-c-socket-server-peerauthentication
namespace: server-c-socket-server
spec:
mtls:
mode: STRICT
<标题>

系统Kubernetes:MicroK8s v1.25.5 revision 4418

kubectl版本:客户端版本:v1.25.5定制版本:v4.5.7服务器版本:v1.25.5

OS:Ubuntu 22.04.1

istioctl替代地位

NAME                                                                      CLUSTER        CDS        LDS        EDS        RDS          ECDS         ISTIOD                     VERSION
istio-ingressgateway-78f69b5b89-w24fx.istio-system                        Kubernetes     SYNCED     SYNCED     SYNCED     NOT SENT     NOT SENT     istiod-d887c9b84-xk9tn     1.14.4
server-c-nginx-deploy-7cb9cc7574-57tdw.server-c-nginx                     Kubernetes     SYNCED     SYNCED     SYNCED     SYNCED       NOT SENT     istiod-d887c9b84-xk9tn     1.14.4
server-c-socket-client-deploy-7469697f89-ndf89.server-c-socket-client     Kubernetes     SYNCED     SYNCED     SYNCED     SYNCED       NOT SENT     istiod-d887c9b84-xk9tn     1.14.4
server-c-socket-server-deploy-5d47669d86-fk8kh.server-c-socket-server     Kubernetes     SYNCED     SYNCED     SYNCED     SYNCED       NOT SENT     istiod-d887c9b84-xk9tn     1.14.4

我已经在istio中尝试了许多属性，并且在创建EnvoyFilter时遇到了困难，不幸的是它没有给出预期的结果。例如:"NR filter_chain_not_found"#30819 https://github.com/istio/istio/issues/30819或https://vikaschoudhary16.com/2022/06/20/undeistio-permissive-authz-magic/#Scenario_2_non-injected_client_to_injected_and_non-injected_services

最后，纯文本消息(TCP)应该加密，这在STRICT模式下不起作用。

如果您有任何想法或需要更多信息，请告诉我。

致以最亲切的问候。

部分更新文件| 01/19/2023:

  • 通信在一个集群
  • 没有出/入外部集群流量(例如。没有配置入口或出口网关)
  • 套接字服务器位于命名空间:Server -c- Socket - Server
  • Socket客户端位于命名空间:server-c-socket-client
  • 如果我编辑PeerAuthentication从Socket服务器到PERMISSIVE它立即工作，但不加密…(
  • 我还向套接字客户端Python脚本添加了一个sleep命令(大约3分钟)，因为我怀疑部署和envoy-sidecar之间存在定时问题
  • 我注意到Envoy "10.1.2.142:50000 10.1.2.146:50001"第一个ip地址是套接字服务器，第二个是套接字客户端，它看起来像服务器不知道如何回复套接字连接请求…

istio-destinationrule-socket-client.yaml

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: server-c-socket-client-destinationrule
namespace: server-c-socket-client
spec:
host: server-c-socket-client-service.server-c-socket-client.svc.cluster.local
subsets:
- name: v1
labels:
version: v1
trafficPolicy:
tls:
mode: ISTIO_MUTUAL
sni: server-c-socket-client-service.server-c-socket-client.svc.cluster.local

istio-destinationrule-socket-server.yaml

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: server-c-socket-server-destinationrule
namespace: server-c-socket-server
spec:
host: server-c-socket-server-service.server-c-socket-server.svc.cluster.local
subsets:
- name: v1
labels:
version: v1
trafficPolicy:
tls:
mode: ISTIO_MUTUAL
sni: server-c-socket-server-service.server-c-socket-server.svc.cluster.local

istio-peerauthentication-socket-server.yaml

apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
name: server-c-socket-server-peerauthentication
namespace: server-c-socket-server
spec:
mtls:
mode: STRICT

istio-peerauthentication-socket-client.yaml

apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
name: server-c-socket-client-peerauthentication
namespace: server-c-socket-client
spec:
mtls:
mode: STRICT

istio-strict-meshpolicy.yaml

apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
name: default
namespace: istio-system
spec:
mtls:
mode: STRICT

istio-virtualservice-socket-client.yaml

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: server-c-socket-client-virtualservice
namespace: server-c-socket-client
spec:
hosts:
- server-c-socket-client-service.server-c-socket-client.svc.cluster.local
tcp:
- match:
- port: 50001
route:
- destination:
host: server-c-socket-client-service.server-c-socket-client.svc.cluster.local
subset: v1
port:
number: 50001
weight: 100

istio-virtualservice-socket-server.yaml

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: server-c-socket-server-virtualservice
namespace: server-c-socket-server
spec:
hosts:
- server-c-socket-server-service.server-c-socket-server.svc.cluster.local
tcp:
- match:
- port: 50000
route:
- destination:
host: server-c-socket-server-service.server-c-socket-server.svc.cluster.local
subset: v1
port:
number: 50000
weight: 100

istio-protocolversion.yaml

apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
meshConfig:
enableTracing: true
accessLogFile: "/dev/stdout"
meshMTLS:
minProtocolVersion: TLSV1_3

server-c@server-c:~$ microk8s istioctl experimental describe pod server-c-socket-client-deploy-7469697f89-ngktr.server-c-socket-client
Pod: server-c-socket-client-deploy-7469697f89-ngktr.server-c-socket-client
Pod Revision: default
Pod Ports: 50001 (server-c-socket-client-app), 15090 (istio-proxy)
WARNING: User ID (UID) 1337 is reserved for the sidecar proxy.
--------------------
Service: server-c-socket-client-service.server-c-socket-client
Port: tcp 50001/TCP targets pod port 50001
DestinationRule: server-c-socket-client-destinationrule.server-c-socket-client for "server-c-socket-client-service.server-c-socket-client.svc.cluster.local"
Matching subsets: v1
Traffic Policy TLS Mode: ISTIO_MUTUAL
--------------------
Effective PeerAuthentication:
Workload mTLS mode: STRICT
Applied PeerAuthentication:
default.istio-system, server-c-socket-client-peerauthentication.server-c-socket-client

server-c@server-c:~$ microk8s istioctl experimental describe pod server-c-socket-server-deploy-5d47669d86-s9wzj.server-c-socket-server
Pod: server-c-socket-server-deploy-5d47669d86-s9wzj.server-c-socket-server
Pod Revision: default
Pod Ports: 50000 (server-c-socket-server-app), 15090 (istio-proxy)
WARNING: User ID (UID) 1337 is reserved for the sidecar proxy.
--------------------
Service: server-c-socket-server-service.server-c-socket-server
Port: tcp 50000/TCP targets pod port 50000
DestinationRule: server-c-socket-server-destinationrule.server-c-socket-server for "server-c-socket-server-service.server-c-socket-server.svc.cluster.local"
Matching subsets: v1
Traffic Policy TLS Mode: ISTIO_MUTUAL
--------------------
Effective PeerAuthentication:
Workload mTLS mode: STRICT
Applied PeerAuthentication:
default.istio-system, server-c-socket-server-peerauthentication.server-c-socket-server

mtl:严格的

server-c@server-c:~$ microk8s istioctl pc listeners deploy/server-c-socket-server-deploy -n server-c-socket-server --port 15006
ADDRESS         PORT    MATCH                                                                                       DESTINATION
0.0.0.0         15006   Addr: *:15006                                                                               Non-HTTP/Non-TCP
0.0.0.0         15006   Trans: tls; App: istio-http/1.0,istio-http/1.1,istio-h2; Addr: 0.0.0.0/0                    InboundPassthroughClusterIpv4
0.0.0.0         15006   Trans: tls; Addr: 0.0.0.0/0                                                                 InboundPassthroughClusterIpv4
0.0.0.0         15006   Trans: tls; Addr: *:50000                                                                   Cluster: inbound|50000||

mtl:宽容的

server-c@server-c:~$ microk8s istioctl pc listeners deploy/server-c-socket-server-deploy -n server-c-socket-server --port 15006
ADDRESS         PORT  MATCH                                                                                                         DESTINATION
0.0.0.0         15006   Addr: *:15006                                                                                               Non-HTTP/Non-TCP
0.0.0.0         15006   Trans: tls; App: istio-http/1.0,istio-http/1.1,istio-h2; Addr: 0.0.0.0/0                                    InboundPassthroughClusterIpv4
0.0.0.0         15006   Trans: raw_buffer; App: http/1.1,h2c; Addr: 0.0.0.0/0                                                       InboundPassthroughClusterIpv4
0.0.0.0         15006   Trans: tls; App: TCP TLS; Addr: 0.0.0.0/0                                                                   InboundPassthroughClusterIpv4
0.0.0.0         15006   Trans: raw_buffer; Addr: 0.0.0.0/0                                                                          InboundPassthroughClusterIpv4
0.0.0.0         15006   Trans: tls; Addr: 0.0.0.0/0                                                                                 InboundPassthroughClusterIpv4
0.0.0.0         15006   Trans: tls; App: istio,istio-peer-exchange,istio-http/1.0,istio-http/1.1,istio-h2; Addr: *:50000            Cluster: inbound|50000||
0.0.0.0         15006   Trans: tls; Addr: *:50000                                                                                   Cluster: inbound|50000||
0.0.0.0         15006   Trans: raw_buffer; Addr: *:50000                                                                            Cluster: inbound|50000||

我太关注Istio了，没有进一步研究底层应用程序…

在我之前的应用程序中，服务器试图直接与客户端pod通信，而不是通过套接字客户端服务(这是基本问题)。我已经改变了我的应用程序，以便有一个套接字客户端和一个套接字服务器在双方。这些是交替产生的，所以第一个客户端A联系服务器B，然后客户端B联系服务器A =>总是交替。这意味着我现在可以在Istio中跟踪两个服务的连通性，然后用mTLS尝试它们。工作套接字客户端和服务端=>mtl

相关内容

最新更新


热门标签：

javascript python java c# php android html jquery c++ css ios sql mysql arrays asp.net json python-3.x ruby-on-rails .net sql-server django objective-c excel regex ruby linux ajax iphone xml vba spring asp.net-mvc database wordpress string postgresql wpf windows xcode bash git oracle list vb.net multithreading eclipse algorithm macos powershell visual-studio image forms numpy scala function api selenium