即使使用率低于阈值，Kubernetes部署也不会缩减 [英] Kubernetes deployment not scaling down even though usage is below threshold

问题描述

我很难理解水平吊舱自动缩放器的状况.

I’m having a hard time understanding what’s going on with my horizontal pod autoscaler.

如果内存或CPU使用率超过80％，我将尝试扩大部署范围.

I’m trying to scale up my deployment if the memory or cpu usage goes above 80%.

这是我的HPA模板:

apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata:
  name: my-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: my-deployment
  minReplicas: 2
  maxReplicas: 10
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 80
    - type: Resource
      resource:
        name: memory
        target:
          type: Utilization
          averageUtilization: 80

问题是，即使使用率低于80％，它也被放置了3天，但我不明白为什么.

The thing is, it’s been sitting at 3 replicas for days even though the usage is below 80% and I don’t understand why.

$ kubectl get hpa --all-namespaces

NAMESPACE        NAME             REFERENCE                  TARGETS            MINPODS   MAXPODS   REPLICAS   AGE
my-ns            my-hpa           Deployment/my-deployment   61%/80%, 14%/80%   2         10        3          2d15h

这是top命令的输出:

Here’s the output of the top command:

$ kubectl top pods

NAME                             CPU(cores)   MEMORY(bytes)   
my-deployment-86874588cc-chvxq   3m           146Mi           
my-deployment-86874588cc-gkbg9   5m           149Mi           
my-deployment-86874588cc-nwpll   7m           149Mi   

每个Pod消耗大约60％的请求内存(因此它们低于80％的目标):

Each pod consumes approximately 60% of their requested memory (So they are below the 80% target):

resources:
  requests:
    memory: "256Mi"
    cpu: "100m"
  limits:
    memory: "512Mi"
    cpu: "200m"

这是我的部署:

kind: Deployment
apiVersion: apps/v1
metadata:
  name: my-deployment
  labels:
    app: my-app
spec:
  replicas: 2
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
        - name: my-app
          image: ...
          imagePullPolicy: Always
          resources:
            requests:
              memory: "256Mi"
              cpu: "100m"
            limits:
              memory: "512Mi"
              cpu: "200m"
          livenessProbe:
            httpGet:
              path: /liveness
              port: 3000
            initialDelaySeconds: 10
            periodSeconds: 3
            timeoutSeconds: 3
          readinessProbe:
            httpGet:
              path: /readiness
              port: 3000
            initialDelaySeconds: 10
            periodSeconds: 3
            timeoutSeconds: 3
          ports:
            - containerPort: 3000
              protocol: TCP

我手动缩小到2个副本，并且立即无缘无故地回升到3个副本:

I manually scale down to 2 replicas and it goes back up to 3 right away for no reason:

Normal   SuccessfulRescale             28s (x4 over 66m)    horizontal-pod-autoscaler  New size: 3; reason:

任何人都知道发生了什么事吗?

Anyone have any idea what’s going on?

推荐答案

https://kubernetes.io/docs/tasks/run-application/horizo​​ntal-pod-autoscale/#algorithm-details

根据您当前的数字，除非您的内存使用量降低到所需百分比的一半，否则它永远不会缩小.

As per your current numbers, It will never scale down unless your memory usage goes down to half of the desired percentage.

即cpu和内存的当前利用率都应达到40％(在您的情况下)或更低

i.e the current utilization of both cpu and memory should go to 40%(in your case) or below

按照以下公式

desiredReplicas = ceil[currentReplicas * ( currentMetricValue / desiredMetricValue )]
                = ceil[3 * (61/80)]
                = ceil[3 * (0.7625)]
                = ceil[2.2875]
desiredReplicas = 3

您可能会怀疑cpu低于40％为何不进行降级..但是hpa无法以这种方式工作..它将始终寻找更大的数字.

you might be having doubt like your cpu is below 40% why it is not downscaling.. but hpa will not work in that way.. it will always look for larger number.

这篇关于即使使用率低于阈值，Kubernetes部署也不会缩减的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！