使用kubespray的本地Kubernetes kubelet-certificate-authority导致主节点的证书验证错误 [英] Kubernetes kubelet-certificate-authority on premise with kubespray causes certificate validation error for master node

问题描述

我正在使用kubespray在内部设置k8s集群.

I'm setting up a k8s cluster on premise using kubespray.

我正在尝试使用CIS Benchmark文档强化kubernetes集群.

I'm trying to harden the kubernetes cluster using CIS Benchmark documentation.

对于-kubelet-certificate-authority 参数，我在apiserver和kubelet之间建立了TLS连接.然后，我在主节点上编辑了API服务器pod规范文件/etc/kubernetes/manifests/kube-apiserver.yaml 并设置了-kubelet-certificate-authority 证书颁发机构的参数，例如:-kubelet-certificate-authority =/etc/kubernetes/ssl/apiserver.crt

For the --kubelet-certificate-authority argument I set up the TLS connection between the apiserver and kubelets. Then, I edited the API server pod specification file /etc/kubernetes/manifests/kube-apiserver.yaml on the master node and set the --kubelet-certificate-authority parameter for the certificate authority like this : --kubelet-certificate-authority=/etc/kubernetes/ssl/apiserver.crt

但是，由于已知错误，我不再能够(使用头盔)进行顶部部署吊舱:

But with that I'm not longer able top deploy pods (using helm), having the known error :

[centos @ infra-vm〜] $掌舵清单错误:转发端口:错误升级连接:错误拨号后端:x509:无法验证192.168.33.143的证书，因为它不包含任何IP SAN

[centos@infra-vm ~]$ helm list Error: forwarding ports: error upgrading connection: error dialing backend: x509: cannot validate certificate for 192.168.33.143 because it doesn't contain any IP SANs

其中 192.168.33.143 是主节点IP地址.

Where 192.168.33.143 is the master node IP address.

我已经检查了上面的证书资格，并且具有IP SAN:

I've checked the above certificate autority and is has IP SANs :

所以我真的不知道问题出在哪里.

So I really can't figure out where the issue comes from .

[centos@infra-vm ~]$ kubectl get pod --namespace kube-system
NAME                                        READY   STATUS    RESTARTS   AGE
calico-kube-controllers-7555c9885c-tjz78    1/1     Running   0          3d21h
calico-node-2p4p4                           1/1     Running   0          3d21h
calico-node-4rhzj                           1/1     Running   0          3d21h
coredns-56bc6b976d-wrxsl                    1/1     Running   0          3d21h
coredns-56bc6b976d-zlvxb                    1/1     Running   0          3d21h
dns-autoscaler-5fc5fdbf6-sl6gg              1/1     Running   0          3d21h
kube-apiserver-cpu-node0                    1/1     Running   0          3d21h
kube-controller-manager-cpu-node0           1/1     Running   0          3d21h
nvidia-device-plugin-daemonset-1.12-zj82x   1/1     Running   0          3d20h
tiller-deploy-677fbf76bb-hcgtw              1/1     Running   0          3d21h

[centos@infra-vm ~]$ kubectl logs tiller-deploy-677fbf76bb-hcgtw  --namespace kube-system
Error from server: Get https://192.168.33.143:10250/containerLogs/kube-system/tiller-deploy-677fbf76bb-hcgtw/tiller: x509: cannot validate certificate for 192.168.33.143 because it doesn't contain any IP SANs
[centos@infra-vm ~]$

有人可以帮忙弄清楚到底是怎么回事吗?

Could one try to help figure out what is going on?

推荐答案

首先/etc/kubernetes/ssl/apiserver.crt 不是有效的CA证书.CA将具有:

First of all /etc/kubernetes/ssl/apiserver.crt is not a valid CA certificate. CA would have:

    X509v3 extensions:
        X509v3 Key Usage: critical
            Digital Signature, Key Encipherment, Certificate Sign

通知证书签名扩展名，允许对证书进行签名.

Notice Certificate Sign extension that allows for signing certificates.

您看到此错误:无法验证192.168.33.143的证书，因为它不包含任何IP SAN ，因为kubelet使用自签名证书在端口10250上提供https流量，并且您正在使用无效的证书来对其进行验证.

You are seeing this error: cannot validate certificate for 192.168.33.143 because it doesn't contain any IP SANs because kubelet is using self signed certificates to serve https traffic on port 10250 and you are using invalid certificate to validate it.

那你应该怎么做才能使它工作呢?

So what should you do to make it work??

• 使用/etc/kubernetes/ssl/ca.crt 为具有有效IP SAN的kubelet签署新证书.
• 设置-kubelet-certificate-authority =/etc/kubernetes/ssl/ca.crt (有效CA).
• 在/var/lib/kubelet/config.yaml (kubelet配置文件)中，设置 tlsCertFile 和 tlsPrivateKeyFile 指向新创建的kubelet crt 和 key 文件.

• Use /etc/kubernetes/ssl/ca.crt to sign new certificate for kubelet with valid IP SANs.
• Set --kubelet-certificate-authority=/etc/kubernetes/ssl/ca.crt (valid CA).
• In /var/lib/kubelet/config.yaml (kubelet config file) set tlsCertFile and tlsPrivateKeyFile to point to newly created kubelet crt and key files.

这篇关于使用kubespray的本地Kubernetes kubelet-certificate-authority导致主节点的证书验证错误的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！