无法创建需要超过2Gi内存的部署 [英] Cannot create a deployment that requests more than 2Gi memory

问题描述

我的部署吊舱由于内存消耗而被驱逐:

My deployment pod was evicted due to memory consumption:

  Type     Reason   Age   From                                             Message
  ----     ------   ----  ----                                             -------
  Warning  Evicted  1h    kubelet, gke-XXX-default-pool-XXX  The node was low on resource: memory. Container my-container was using 1700040Ki, which exceeds its request of 0.
  Normal   Killing  1h    kubelet, gke-XXX-default-pool-XXX  Killing container with id docker://my-container:Need to kill Pod

我尝试通过将以下内容添加到我的部署yaml中来授予它更多的内存:

I tried to grant it more memory by adding the following to my deployment yaml:

apiVersion: apps/v1
kind: Deployment
...
spec:
  ...
  template:
    ...
    spec:
      ...
      containers:

      - name: my-container
        image: my-container:latest
        ...
        resources:
          requests:
            memory: "3Gi"

但是，它部署失败:

  Type     Reason             Age               From                Message
  ----     ------             ----              ----                -------
  Warning  FailedScheduling   4s (x5 over 13s)  default-scheduler   0/3 nodes are available: 3 Insufficient memory.
  Normal   NotTriggerScaleUp  0s                cluster-autoscaler  pod didn't trigger scale-up (it wouldn't fit if a new node is added)

部署仅请求一个容器.

我正在使用GKE进行自动缩放，默认(且唯一)池中的节点具有3.75 GB内存.

I'm using GKE with autoscaling, the nodes in the default (and only) pool have 3.75 GB memory.

通过反复试验，我发现我可以请求的最大内存为"2Gi".为什么我不能在单个Pod中使用节点的全部3.75?我需要更大内存的节点吗?

From trial and error, I found that the maximum memory I can request is "2Gi". Why can't I utilize the full 3.75 of a node with a single pod? Do I need nodes with bigger memory capacity?

推荐答案

即使该节点具有3.75 GB的总内存，也很可能可分配的容量并非全部为3.75 GB.

Even though the node has 3.75 GB of total memory, is very likely that the capacity allocatable is not all 3.75 GB.

Kubernetes为系统服务保留一些容量，以避免容器在节点中消耗过多的资源而影响系统服务的运行.

Kubernetes reserve some capacity for the system services to avoid containers consuming too much resources in the node affecting the operation of systems services .

从文档:

Kubernetes节点可以安排为容量".默认情况下，Pod可以消耗节点上的所有可用容量 .这是一个问题，因为节点通常运行许多为操作系统和Kubernetes本身供电的系统守护程序.除非为这些系统守护程序留出资源，否则pod和系统守护程序会争用资源，并导致节点上出现资源匮乏的问题.

Kubernetes nodes can be scheduled to Capacity. Pods can consume all the available capacity on a node by default. This is an issue because nodes typically run quite a few system daemons that power the OS and Kubernetes itself. Unless resources are set aside for these system daemons, pods and system daemons compete for resources and lead to resource starvation issues on the node.

因为您使用的是GKE，是因为它们没有使用默认值，所以运行以下命令将显示节点中有多少可分配资源:

Because you are using GKE, is they don't use the defaults, running the following command will show how much allocatable resource you have in the node:

kubectl describe node [NODE_NAME] | grep Allocatable -B 4 -A 3

来自 GKE文档:

可分配资源的计算方式如下:

Allocatable resources are calculated in the following way:

可分配=容量-保留-驱逐阈值

Allocatable = Capacity - Reserved - Eviction Threshold

对于内存资源，GKE保留以下内容:

For memory resources, GKE reserves the following:

• 前4GB内存的25％
• 接下来的4GB内存的20％(最多8GB)
• 接下来的8GB内存的10％(最高16GB)
• 接下来的112GB内存(最多128GB)的6％
• 128 GB以上的所有内存的2％

• 25% of the first 4GB of memory
• 20% of the next 4GB of memory (up to 8GB)
• 10% of the next 8GB of memory (up to 16GB)
• 6% of the next 112GB of memory (up to 128GB)
• 2% of any memory above 128GB

GKE在每个节点上保留了额外的100 MiB内存，用于逐出kubelet.

GKE reserves an additional 100 MiB memory on each node for kubelet eviction.

如错误消息所暗示的那样，扩展群集将无法解决问题，因为每个节点的容量限制为X内存量，而POD则需要更多.

As the error message suggests, scaling the cluster will not solve the problem because each node capacity is limited to X amount of memory and the POD need more than that.

这篇关于无法创建需要超过2Gi内存的部署的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！