Kubernetes命名空间处于终止状态 [英] Kubernetes Namespaces stuck in Terminating status

问题描述

我们有一个集群，其中似乎永远都不想完全删除名称空间，现在无法重新创建自定义指标名称空间以能够收集自定义指标以正确设置HPA.我完全理解，我可以使用所有自定义指标资源创建另一个名称空间，但考虑到名称空间陷入终止"状态，则有点担心群集的整体运行状况

We have one cluster where it seems that namespaces never want to be deleted completely and now can't re-create custom-metrics namespace to be able to collect custom metrics to properly setup HPA. I fully understand that I can create another namespace with all custom-metrics resources, but a little concerned with the overall health of the cluster, given that the namespaces get stuck in "Terminating" state

$ kubectl get ns
NAME             STATUS        AGE
cert-manager     Active        14d
custom-metrics   Terminating   7d
default          Active        222d
nfs-share        Active        15d
ingress-nginx    Active        103d
kube-public      Active        222d
kube-system      Active        222d
lb               Terminating   4d
monitoring       Terminating   6d
production       Active        221d

我已经尝试将名称空间导出为JSON，删除终结器并使用已编辑的JSON文件重新创建.还尝试kubectl编辑ns自定义指标并删除-kubernetes"终结器.全部无济于事.

I already tried to export namespaces to JSON, delete finalizers and re-create using edited JSON files. also tried to kubectl edit ns custom-metrics and delete "- kubernetes" finalizer. all to no avail.

有人对我如何尝试销毁这些卡住"的命名空间有其他建议吗?

does anyone have any other recommendations on how else I can try to destroy these "stuck" namespaces"

卷曲到 https://master-ip/api/v1 /namespace/...../finalize 对于我来说似乎不适用于Google Kubernetes Engine，我假设在GKE集群上不允许执行这些操作

curl to https://master-ip/api/v1/namespace/...../finalize doesn't seem to work on Google Kubernetes Engine for me, I'm assuming these operations are not allowed on GKE cluster

尝试类似的方法也不起作用:

Trying things like doesn't work as well:

$ kubectl delete ns custom-metrics --grace-period=0 --force

警告:立即删除不会等待确认 正在运行的资源已终止.资源可能会继续运行 无限期地在群集上.来自服务器的错误(冲突):操作 无法在名称空间"custom-metrics"上实现:系统为 确保从此名称空间中删除所有内容.完成后， 该名称空间将由系统自动清除.

warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. Error from server (Conflict): Operation cannot be fulfilled on namespaces "custom-metrics": The system is ensuring all content is removed from this namespace. Upon completion, this namespace will automatically be purged by the system.

，此命名空间中根本没有列出资源: kubectl get all -n custom-metrics或循环访问此命名空间中的所有api资源都表明根本不存在任何资源: kubectl api-resources --namespaced=true -o name | xargs -n 1 kubectl get -n custom-metrics

and there're no resources listed in this namespaces at all: kubectl get all -n custom-metrics or looping through all api-resources in this namespace shows no resources exist at all: kubectl api-resources --namespaced=true -o name | xargs -n 1 kubectl get -n custom-metrics

推荐答案

像这样的已知问题，人们在尝试混合不同结果时会有不同的结果:

Looks like this is a known issue with people having mixed results trying a mix of different things:

• 弹跳kube-controller-manager
• 弹跳所有kubelets
• 弹跳整个集群
• kubectl delete ns <name> --grace-period=0 --force
• 修补终结器到处都是空".

• Bounce the kube-controller-manager
• Bounce all the kubelets
• Bounce the whole cluster
• kubectl delete ns <name> --grace-period=0 --force
• Patching finalizers "null" everywhere.

更多背景，但在广告连播级别上也在这里.

Some more background but at the pod level here too.

这篇关于Kubernetes命名空间处于终止状态的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！