服务"kube-dns"无效:spec.clusterIP:无效值:"10.10.0.10&":字段不可变 [英] Service "kube-dns" is invalid: spec.clusterIP: Invalid value: "10.10.0.10": field is immutable

查看：3495
本文介绍了服务"kube-dns"无效:spec.clusterIP:无效值:"10.10.0.10&":字段不可变的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！

问题描述

我通过kubeadm设置了集群.在最后一步，我执行kubeadm init --config kubeadm.conf --v=5.我收到有关clusterIp值的错误.这是输出的一部分:

I setup my cluster by kubeadm. At the last step i exec kubeadm init --config kubeadm.conf --v=5. I get the error about the clusterIp value. Here is the part of the output: 

    I0220 00:16:27.625920   31630 clusterinfo.go:79] creating the RBAC rules for exposing the cluster-info ConfigMap in the kube-public namespace
I0220 00:16:27.947941   31630 kubeletfinalize.go:88] [kubelet-finalize] Assuming that kubelet client certificate rotation is enabled: found "/var/lib/kubelet/pki/kubelet-client-current.pem"
[kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
I0220 00:16:27.949398   31630 kubeletfinalize.go:132] [kubelet-finalize] Restarting the kubelet to enable client certificate rotation
[addons]: Migrating CoreDNS Corefile
I0220 00:16:28.447420   31630 dns.go:381] the CoreDNS configuration has been migrated and applied: .:53 {
    errors
    health {
       lameduck 5s
    }
    ready
    kubernetes cluster.local in-addr.arpa ip6.arpa {
       pods insecure
       fallthrough in-addr.arpa ip6.arpa
       ttl 30
    }
    prometheus :9153
    forward . /etc/resolv.conf
    cache 30
    loop
    reload
    loadbalance
}
.
I0220 00:16:28.447465   31630 dns.go:382] the old migration has been saved in the CoreDNS ConfigMap under the name [Corefile-backup]
I0220 00:16:28.447486   31630 dns.go:383] The changes in the new CoreDNS Configuration are as follows:
Service "kube-dns" is invalid: spec.clusterIP: Invalid value: "10.10.0.10": field is immutable
unable to create/update the DNS service
k8s.io/kubernetes/cmd/kubeadm/app/phases/addons/dns.createDNSService
    /workspace/anago-v1.17.0-rc.2.10+70132b0f130acc/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubeadm/app/phases/addons/dns/dns.go:323
k8s.io/kubernetes/cmd/kubeadm/app/phases/addons/dns.createCoreDNSAddon
    /workspace/anago-v1.17.0-rc.2.10+70132b0f130acc/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubeadm/app/phases/addons/dns/dns.go:305
k8s.io/kubernetes/cmd/kubeadm/app/phases/addons/dns.coreDNSAddon

我的配置文件是这样的:

And my config file like this:

apiVersion: kubeadm.k8s.io/v1beta2
bootstrapTokens:
- groups:
  - system:bootstrappers:kubeadm:default-node-token
  token: abcdef.0123456789abcdef
  ttl: 24h0m0s
  usages:
  - signing
  - authentication
kind: InitConfiguration
localAPIEndpoint:
  advertiseAddress: 172.16.5.151
  bindPort: 6443
nodeRegistration:
  criSocket: /var/run/dockershim.sock
  name: master02
#  taints:
#  - effect: NoSchedule
#    key: node-role.kubernetes.io/master
---
apiServer:
  certSANs:
    - "172.16.5.150"
    - "172.16.5.151"
    - "172.16.5.152"
  timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta2
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controllerManager: {}
dns:
  type: CoreDNS
etcd:
  external:
    endpoints:
    - "https://172.16.5.150:2379"
    - "https://172.16.5.151:2379"
    - "https://172.16.5.152:2379"
    caFile: /etc/k8s/pki/ca.pem
    certFile: /etc/k8s/pki/etcd.pem
    keyFile: /etc/k8s/pki/etcd.key

imageRepository: registry.aliyuncs.com/google_containers
kind: ClusterConfiguration
kubernetesVersion: v1.17.0
networking:
  dnsDomain: cluster.local
  serviceSubnet: 10.10.0.0/16
  podSubnet: 192.168.0.0/16
scheduler: {}

我检查了kubeadm生成的kube-apiserver.yaml. --service-cluster-ip-range = 10.10.0.0/16设置包含10.10.0.10 您可以在下面看到:

I checked the kube-apiserver.yaml generated by kubeadm. the --service-cluster-ip-range=10.10.0.0/16 settings is contains 10.10.0.10 you can see below:

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    component: kube-apiserver
    tier: control-plane
  name: kube-apiserver
  namespace: kube-system
spec:
  containers:
  - command:
    - kube-apiserver
    - --advertise-address=172.16.5.151
    - --allow-privileged=true
    - --authorization-mode=Node,RBAC
    - --client-ca-file=/etc/kubernetes/pki/ca.crt
    - --enable-admission-plugins=NodeRestriction
    - --enable-bootstrap-token-auth=true
    - --etcd-cafile=/etc/k8s/pki/ca.pem
    - --etcd-certfile=/etc/k8s/pki/etcd.pem
    - --etcd-keyfile=/etc/k8s/pki/etcd.key
    - --etcd-servers=https://172.16.5.150:2379,https://172.16.5.151:2379,https://172.16.5.152:2379
    - --insecure-port=0
    - --kubelet-client-certificate=/etc/kubernetes/pki/apiserver-kubelet-client.crt
    - --kubelet-client-key=/etc/kubernetes/pki/apiserver-kubelet-client.key
    - --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname
    - --proxy-client-cert-file=/etc/kubernetes/pki/front-proxy-client.crt
    - --proxy-client-key-file=/etc/kubernetes/pki/front-proxy-client.key
    - --requestheader-allowed-names=front-proxy-client
    - --requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-ca.crt
    - --requestheader-extra-headers-prefix=X-Remote-Extra-
    - --requestheader-group-headers=X-Remote-Group
    - --requestheader-username-headers=X-Remote-User
    - --secure-port=6443
    - --service-account-key-file=/etc/kubernetes/pki/sa.pub
    - --service-cluster-ip-range=10.10.0.0/16
    - --tls-cert-file=/etc/kubernetes/pki/apiserver.crt
    - --tls-private-key-file=/etc/kubernetes/pki/apiserver.key
    image: registry.aliyuncs.com/google_containers/kube-apiserver:v1.17.0
    imagePullPolicy: IfNotPresent
    livenessProbe:
      failureThreshold: 8
      httpGet:
        host: 172.16.5.151
        path: /healthz
        port: 6443
        scheme: HTTPS
      initialDelaySeconds: 15
      timeoutSeconds: 15
    name: kube-apiserver
    resources:
      requests:
        cpu: 250m
    volumeMounts:
    - mountPath: /etc/ssl/certs
      name: ca-certs
      readOnly: true
    - mountPath: /etc/pki
      name: etc-pki
      readOnly: true
    - mountPath: /etc/k8s/pki
      name: etcd-certs-0
      readOnly: true
    - mountPath: /etc/kubernetes/pki
      name: k8s-certs
      readOnly: true
  hostNetwork: true
  priorityClassName: system-cluster-critical
  volumes:
  - hostPath:
      path: /etc/ssl/certs
      type: DirectoryOrCreate
    name: ca-certs
  - hostPath:
      path: /etc/pki
      type: DirectoryOrCreate
    name: etc-pki
  - hostPath:
      path: /etc/k8s/pki
      type: DirectoryOrCreate
    name: etcd-certs-0
  - hostPath:
      path: /etc/kubernetes/pki
      type: DirectoryOrCreate
    name: k8s-certs
status: {}

如上所示.所有service-ip-range都已设置为10.10.0.0/16.奇怪的是，当我执行"kubectl get svc"时，我得到的kubernetes clusterip是10.96.0.1

As you see above. all the service-ip-range has been set to 10.10.0.0/16. It is strange that when i exec "kubectl get svc" I get the kubernetes clusterip is 10.96.0.1

[root@master02 manifests]# kubectl get svc
NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   2d3h

表示默认的service-ip-range是:10.96.0.0/16.而且我修改的内容行不通.有谁知道如何自定义service-ip-range范围.以及如何解决我的问题?

which means the default service-ip-range is: 10.96.0.0/16. And what i modifyed does not work. Does anyone know How to Custom the service-ip-range scope. And how to slove my problem?

推荐答案

将此答案发布为社区Wiki，以扩展并解释根本原因.

Posting this answer as community wiki to expand and explain root cause.

启动kubeadm时，如果我们未指定任何标志，则$ kubeadm init它将创建具有默认值的kubeadm群集.您可以检入 Kubernetes文档标志，该标志可以在初始化期间指定，这是默认值.

When kubeadm is initiated, and we do not specify any flags, $ kubeadm init it will create kubeadm cluster with default values. You can check in Kubernetes docs flags which can be specified during initilization and which are default values.

--service-cidr字符串默认:"10.96.0.0/12" 为服务VIP使用其他IP地址范围.

--service-cidr string Default: "10.96.0.0/12" Use alternative range of IP address for service VIPs.

这就是默认kubernetes服务将10.96.0.1用作ClusterIP的原因.

That's the reason why default kubernetes service used 10.96.0.1 as ClusterIP.

这里OP也想使用自己的配置.

Here OP also wanted to use own config.

--config字符串kubeadm配置文件的路径.

--config string Path to a kubeadm configuration file.

可以在此处.

Kubernetes文档示例 Kubeadm重置

As Kubernetes docs exmplain Kubeadm reset

执行尽最大努力恢复kubeadm init或kubeadm联接所做的更改.

Performs a best effort revert of changes made by kubeadm init or kubeadm join.

有时取决于我们的配置，某些配置保留在群集中.

Depends on our configuration sometimes, some configs stay on the cluster.

问题，提到遇到的OP

Issue, that OP encountered was mentioned here - External etcd clean up

如果使用外部etcd，则

kubeadm reset不会删除任何etcd数据.这意味着，如果您使用相同的etcd端点再次运行kubeadm init，您将看到先前集群的状态.

kubeadm reset will not delete any etcd data if external etcd is used. This means that if you run kubeadm init again using the same etcd endpoints, you will see state from previous clusters.

关于不可变字段:Service kube-dns is invalid: spec.clusterIP: Invalid value: 10.10.0.10: field is immutable. 在Kubernetes中，保护了某些字段以防止可能会破坏集群工作的更改.

Regarding Immutable fields: Service "kube-dns" is invalid: spec.clusterIP: Invalid value: "10.10.0.10": field is immutable. In Kubernetes, some fields are secured to prevent changes that might disrupt working of the cluster.

如果任何字段是immutable，但我们必须对其进行更改，则必须删除该对象并再次添加.

If any field is immutable but we have to change it, this object must be removed and add again.

这篇关于服务"kube-dns"无效:spec.clusterIP:无效值:"10.10.0.10&":字段不可变的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！

查看全文
相关文章
其他开发最新文章
热门教程
热门工具
登录 关闭
扫码关注1秒登录
发送“验证码”获取 | 15天全站免登陆