AWS ECS任务内存硬限制和软限制 [英] AWS ECS Task Memory Hard and Soft Limits

问题描述

对于ECS任务定义同时具有硬性和软性内存限制的目的，我感到困惑.

I'm confused about the purpose of having both hard and soft memory limits for ECS task definitions.

IIRC的软限制是调度程序在实例上为运行任务保留的内存量，而硬限制是容器在被谋杀之前可以使用的内存量.

IIRC the soft limit is how much memory the scheduler reserves on an instance for the task to run, and the hard limit is how much memory a container can use before it is murdered.

我的问题是，如果ECS调度程序基于软限制将任务分配给实例，那么您可能会遇到一种情况，即正在使用高于软限制但低于硬限制的内存的任务可能会导致实例超过其最大值内存(假设所有其他任务使用的内存略低于或等于其软限制).

My issue is that if the ECS scheduler allocates tasks to instances based on the soft limit, you could have a situation where a task that is using memory above the soft limit but below the hard limit could cause the instance to exceed its max memory (assuming all other tasks are using memory slightly below or equal to their soft limit).

这正确吗?

谢谢

推荐答案

如果您希望运行的计算工作负载主要是内存限制而不是CPU限制，那么您应该仅使用硬限制，而不是软限制.从文档中:

If you expect to run a compute workload that is primarily memory bound instead of CPU bound then you should use only the hard limit, not the soft limit. From the docs:

必须为容器定义中的memory或memoryReservation中的一个或两个指定非零整数.如果同时指定两者，则内存必须大于memoryReservation.如果指定memoryReservation，则从放置容器的容器实例的可用内存资源中减去该值；否则，将使用内存值.

You must specify a non-zero integer for one or both of memory or memoryReservation in container definitions. If you specify both, memory must be greater than memoryReservation. If you specify memoryReservation, then that value is subtracted from the available memory resources for the container instance on which the container is placed; otherwise, the value of memory is used.

http://docs.aws.amazon.com/AmazonECS/Latest/developerguide/task_definition_parameters.html

通过仅为任务指定硬限制，您可以避免用完内存，因为ECS会停止在实例上放置任务，而docker会杀死所有试图超过硬限制的容器.

By specifying only a hard memory limit for your tasks you avoid running out of memory because ECS stops placing tasks on the instance, and docker kills any containers that try to go over the hard limit.

软内存限制功能是为CPU受限的应用程序而设计的，这些应用程序希望保留少量的最小内存(软限制)，但偶尔允许突发达到硬限制.在这种类型的CPU繁重的工作负载中，您实际上并不太在乎容器的内存使用量的具体值，因为容器在耗尽实例内存之前会耗尽CPU的时间很长，因此您可以根据以下内容放置任务CPU预留和软内存限制.在此设置中，硬限制只是故障保护，以防万一发生失控或内存泄漏.

The soft memory limit feature is designed for CPU bound applications where you want to reserve a small minimum of memory (the soft limit) but allow occasional bursts up to the hard limit. In this type of CPU heavy workload you don't really care about the specific value of memory usage for the containers that much because the containers will run out of CPU long before they exhaust the memory of the instance, so you can place tasks based on CPU reservation and the soft memory limit. In this setup the hard limit is just a failsafe in case something goes out of control or there is a memory leak.

因此，总而言之，您应该使用负载测试来评估您的工作负载，并查看它是倾向于先用尽CPU还是首先消耗内存.如果您受CPU限制，则可以将软内存限制与可选的硬限制一起使用，就像故障保护一样.如果您受内存限制，那么您将只需要使用硬限制而没有软限制.

So in summary you should evaluate your workload using load tests and see whether it tends to run out of CPU first or out of memory first. If you are CPU bound then you can use the soft memory limit with an optional hard limit just as a failsafe. If you are memory bound then you will need to use just the hard limit with no soft limit.

这篇关于AWS ECS任务内存硬限制和软限制的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！