通过Istio入口网关的TLS握手失败(tlsMode = passthrough) [英] TLS handshake through Istio ingress gateway fails (tlsMode=passthrough)

问题描述

从外部客户端到Kubernetes集群中的服务器的TLS握手失败.这是关于了解原因的原因.

A TLS handshake from an external client to a server inside a Kubernetes cluster fails. This is about understanding why.

我已经配置了一个Istio入口网关以通过端口15433上接收的TLS，并将其路由到端口433上的服务器.

I've configured an Istio ingress gateway to pass through TLS received on port 15433, and route it to the server on port 433.

当客户端尝试进行TLS握手时，入口网关日志显示活动，但服务器日志或istio-proxy日志均未显示.

The ingress gateway logs shows activity when the client attempts the TLS handshake, but not the server logs, nor the istio-proxy logs.

TLS客户端:

openssl s_client \ 
        -connect [redacted]-[redacted].us-west-2.elb.amazonaws.com:15443 \ 
        -servername myservice.mynamespace \ 
        -CAfile /path/to/ca.cert \ 
        -cert /path/to/cert.pem \ 
        -key /path/to/cert.key <<< "Q"

日志

CONNECTED(00000006) 
140090868934296:error:140790E5:SSL routines:ssl23_write:ssl handshake failure:s23_lib.c:177: 
--- 
no peer certificate available 
--- 
No client certificate CA names sent 
--- 
SSL handshake has read 0 bytes and written 298 bytes 
--- 
New, (NONE), Cipher is (NONE) 
Secure Renegotiation IS NOT supported 
Compression: NONE 
Expansion: NONE 
No ALPN negotiated 
SSL-Session: 
    Protocol  : TLSv1.2 
    Cipher    : 0000 
    Session-ID:  
    Session-ID-ctx:  
    Master-Key:  
    Key-Arg   : None 
    PSK identity: None 
    PSK identity hint: None 
    SRP username: None 
    Start Time: 1600987862 
    Timeout   : 300 (sec) 
    Verify return code: 0 (ok) 

Istio入口网关日志:

"- - -" 0 - "-" "-" 298 0 1069 - "-" "-" "-" "-" "192.168.101.136:443" outbound|443||myservice.mynamespace.svc.cluster.local 192.168.115.141:42350 192.168.115.141:15443 192.168.125.206:23298 myservice.mynamespace - 

其中192.168.101.136是myservice pod的IP，而192.168.115.141是ingressgateway pod的IP.

where 192.168.101.136 is the IP of the myservice pod and 192.168.115.141 is the IP of the ingressgateway pod.

基于IP，这意味着客户端连接已到达网关，该网关似乎已应用了虚拟服务路由并记录了将其转发到Pod的情况.似乎很正常，只是Pod上的istio-proxy没有显示任何活动，也没有服务器记录日志(尽管服务器没有记录在传输层发生的事情).

Based on the IPs, this means the client connection reached the gateway, the gateway seems to have applied the virtualservice route and logged that it was forwarding this to the pod. Seems normal, except the istio-proxy on the pod shows no activity nor the server logs (though the server doesn't log stuff happening at the transport layer).

由于以下端口转发的TLS握手成功，因此已经为TLS正确配置了服务器的AFAIK:

AFAIK the server is properly configured for TLS as the following port-forwarded TLS handshake succeeds:

kubectl port-forward -n mynamespace service/myservice 4430:443 &
openssl s_client \
        -connect localhost:4430 \
        -CAfile /path/to/ca.cert \ 
        -cert /path/to/cert.pem \ 
        -key /path/to/cert.key <<< "Q"
# I get back a TLS session ID, looks good.

因此，这表明istio的网关或虚拟服务配置存在问题.

So this points to a problem with istio's gateway or virtualservice configuration.

网关:

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: mygateway
  namespace: mynamespace
spec:
  selector:
    istio: ingressgateway
  servers:
    - port:
        number: 15443
        name: tls-passthrough
        protocol: TLS
      tls:
        mode: PASSTHROUGH
      hosts:
      - "*"

虚拟服务:

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: myservice
  namespace: mynamespace
spec:
  hosts:
  - "*" 
  gateways:
  - mygateway
  tls:
    - match:
      - port: 15443
        sniHosts:
        - myservice.mynamespace
      route:
      - destination:
          host: myservice
          port:
            number: 443

Kubernetes服务:

apiVersion: v1
kind: Service
metadata:
  name: myservice
  namespace: mynamespace
  labels:
    app: myservice
spec:
  selector:
    app: myservice
  ports:
    - protocol: TCP
      port: 443
      targetPort: 443
      name: grpc-svc

更新: 实际上，来自客户端的TLS流量确实到达了服务器容器，我通过在服务器容器上执行tcpdump port 443并在运行openssl s_client命令时看到了数据包来确认这一点.不清楚为什么Pod上的istio-proxy没有显示出来，也不能解释握手失败的原因.

UPDATE: Actually the TLS traffic from the client does reach the server pod, I've confirmed this by doing tcpdump port 443 on the server pod and seeing packets when I run the openssl s_client command. Unclear why istio-proxy on the pod didn't show this, that doesn't explain why the handshake fails.

我注意到了别的东西.将-msg标志传递给openssl s_client时，在>>"之后，我什么也看不见.没有<<<"，但是tcpdump显示服务器pod将数据包发送回网关.

I noticed something else. Passing -msg flag to openssl s_client, I see nothing coming back, after ">>>" there's no "<<<", yet the tcpdump shows the server pod sending packets back to the gateway.

推荐答案

我的配置中有2个错误:

There were 2 bugs in my configuration:

我的Kubernetes服务的配置中的

• .不幸的是，我的TCP端口443的名称为grpc-svc，这破坏了TLS直通.将该端口重命名为tcp-svc即可解决问题.

• in the configuration of my Kubernetes service. Unfortunately the name of my TCP port 443 was grpc-svc and that breaks TLS passthrough. Renaming this port to tcp-svc resolves the problem.

我不应该使用入口端口15443，这似乎是为其他目的保留的.在Ingressgateway上打开另一个端口9444，然后按照我在问题中配置端口15443的方式(即配置TLS直通)完全配置网关上的端口9444，然后按照与配置虚拟服务时完全相同的方式配置虚拟服务以路由9444我的问题中的15433路由.

I should not be using ingress port 15443, that seems to be reserved for something else. Opening another port 9444 on the ingressgateway, then configuring port 9444 on the gateway exactly as I was configuring port 15443 in my question (i.e. config for TLS passthrough), and then configuring the virtual service to route 9444 exactly as I was configuring the virtual service route for 15433 in my question.

这两个步骤都允许外部客户端openssl s_client通过入口成功将TLS握手通过TLS握手传递到kubernetes服务.

Doing both of these allows openssl s_client from an external client to succeed a TLS handshake to a kubernetes service via the ingress.

这篇关于通过Istio入口网关的TLS握手失败(tlsMode = passthrough)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！