为什么更多的请求发送到新的(动态)实例而不是常驻实例? [英] Why do more requests go to new (dynamic) instances than to resident instance?

问题描述

在 App Engine 标准环境中拥有一个具有自动缩放功能的 Java 应用.现在缩放配置如下:

Have a Java app with auto-scaling on App Engine Standard Environment. Right now the scaling is configured like this:

<instance-class>F2</instance-class>

<automatic-scaling>
    <min-idle-instances>1</min-idle-instances>

    <!-- ‘automatic’ is the default value. -->
    <max-idle-instances>2</max-idle-instances>

    <!-- ‘automatic’ is the default value. -->
    <min-pending-latency>2000ms</min-pending-latency>

    <max-pending-latency>8000ms</max-pending-latency>
    <max-concurrent-requests>60</max-concurrent-requests>
</automatic-scaling>

刚开始尝试 F2 实例，之前使用的是 F1 实例.无论我如何配置自动缩放，似乎新创建的实例(在负载增加时创建)开始获取所有传入请求，而驻留实例的负载非常轻.

Just started trying the F2 instance, was using F1 instances earlier. No matter how I configure my auto-scaling, it seems like the newly-created instance (created when load increases) starts getting all the incoming requests, while the resident instance sits with a very light load.

这是为什么?当然，我无法实时监控流量(以及它流向哪个实例)，但每次我看故事似乎都是一样的.我在下面提供了一些示例屏幕截图.

Why is this? Of course, I am unable to monitor the traffic (and which instance it goes to) in real time, but every time I look the story seems to be the same. I have included a few sample screenshots below.

在以下情况下，三个实例(与上面的配置略有不同)处于空闲状态，但 GAE 的负载均衡器选择将所有请求发送到具有最高延迟的实例！

and in the following case three instances (this was a slightly different configuration from the one above) are sitting free, but GAE's load balancer chooses to send all requests to the instance with the highest latency!

再举一个例子:这是今天上午 10:15:45 开始的常驻实例的请求日志:

One more example: this is the request log for the resident instance started at 10:15:45 AM today:

以及 10 秒后启动的 动态 实例的请求日志:

and the request log for the dynamic instance that started 10 seconds later:

如您所见，动态实例正在处理所有请求(到目前为止 1889 次)，而居民基本上处于空闲状态(同一时间段内有 7 次).如果不是因为常驻实例似乎在创建新动态实例时销毁并重新创建 ，这仍然是可以的.这意味着在一分钟左右的时间内，所有请求都会看到 10-20 秒的响应时间.

As you can see the dynamic instance is handling all of the requests (1889 so far) while the resident sits essentially idle (7 in the same time period). This would still be OK if not for the fact that the resident instances seem to be destroyed and created anew right around the time new dynamic instances are being created. This means that for a minute or so all requests see 10-20 second response times.

有人可以向我解释如何配置吗?

Can somebody please explain to me how to configure?

这是我想要的:

• 一个空闲实例应该能够在大部分时间处理负载(目前).
• 当收到更多请求时，启动一个额外的实例.当它准备好时，开始将流量转移到它.

• One idle instance should be able to handle the load most of the times (for now).
• When more requests come in, spin up an additional instance. When it is ready, start diverting traffic to it.

我正在尝试以有限的预算运行一个合理负载的网站，因此尽可能接近免费配额是很重要的.

I am trying to run a reasonable-load site on a shoestring budget, so it is important that I try to stay as close to the free quota as possible.

由于两个答案都突出地谈到了热身请求，我想我会在此处列出有关它的详细信息.我正在使用 ServletContextListener 来处理初始化.它执行以下操作(使用 Guava 的 Stopwatch 类，用于我编写/明确调用的代码):

Since both the answers talk about the warmup request prominently, I thought I'd list details about it here. I am using a ServletContextListener to handle the initialization. It does the following (times are gathered using Guava's Stopwatch class and are for the code I have written/am explicitly invoking):

1. 注册 Objectify 实体(1.449 秒)
2. Freemarker 初始化 229 毫秒
3. Firebase 初始化 228.2 毫秒

除此之外，我在 中配置了 Shiro 过滤器、Objectify 过滤器和 Jersey 过滤器(在 Jersey 中，我通过显式注册类而不是为其提供要扫描的包来避免类路径扫描(我认为)web.xml.不使用任何依赖注入来避免类路径扫描.

Other than that I have the Shiro filter, the Objectify filter and Jersey filter (in Jersey I am avoiding classpath scanning (I think) by explicitly registering the classes rather than giving it a package to scan) configured in my web.xml. Not using any dependency injection to avoid classpath scanning.

/_ah/warmup 请求耗时 7.8 秒(从上面的时间开始).但是由一个新启动的动态实例服务的请求需要 10 多秒才能完成，尽管这些相同的调用在两分钟后需要 200-700 毫秒.那么，除了我在 StartupListener 中明确做的事情之外，后台还有什么事情发生?

The /_ah/warmup request took 7.8s (the one from where the above times are taken). But requests being served by a freshly-started dynamic instance whose warmup has already finished are taking 10+ seconds to complete, despite the fact that these same calls take 200-700ms two minutes later. So what else is going on in the background other than the stuff I am explicitly doing in my StartupListener?

这是日志的第 1 部分和第 2 部分 日志.

推荐答案

idle instance(s) 的作用不是处理通常的流量，而是能够处理溢出 - 已经运行的临时流量高峰动态实例(如果有)无法处理，直到启动新实例.

The role of the idle instance(s) is not to handle the usual traffic, but to be able to handle overflows - temporary peaks of traffic that the already running dynamic instances (if any) can't handle, until new instances are being started.

从某种意义上说，这就是他们被称为空闲的原因——大多数时候他们只是闲着.恕我直言，在预算压力下，删除空闲实例是首先要做的事情之一.

In a sense that's why they're called idle - most of the time they're just sitting idle. IMHO dropping idle instances is one of the first things to do when on budget pressure.

也可能相关:在谷歌应用引擎中只有一个实例处理大部分请求

旁注:并不是GAE 的负载均衡器选择将所有请求发送到具有最高延迟的实例！.实际上，该首选实例的延迟最高，因为它获得了大部分流量.

Side note: it's not that the GAE's load balancer chooses to send all requests to the instance with the highest latency!. Actually the latency on that preferred instance is the highest because it's the one getting the majority of the traffic.

要防止 GAE 在新实例准备好处理它们之前将流量发送到它们，您需要配置(并正确处理)预热请求:

To prevent GAE from sending traffic to new instances before they're ready to handle them you need to configure (and properly handle) warmup requests:

预热请求是一种特定类型的加载请求应用程序代码提前进入实例，在任何活动之前提出请求.要了解有关如何使用预热请求的更多信息，请参阅预热请求.手动或基本扩展实例不接收/_ah/warmup 请求.

Warmup requests are a specific type of loading request that load application code into an instance ahead of time, before any live requests are made. To learn more about how to use warmup requests, see Warmup Requests. Manual or basic scaling instances do not receive an /_ah/warmup request.

您可能还想考虑一下这个答案:如何为 Google App Engine 保持一个正在运行的实例.基本上是通过 cron 作业防止动态实例保持空闲时间过长，从而使动态实例无限期地运行.

You may also want to give this answer a thought: How do you keep a running instance for Google App Engine. Basically try to keep the dynamic instances running indefinitely by preventing them from remaining idle for too long via a cron job.

至于在新的动态实例启动后常驻实例的明显重启似乎有点奇怪.我不会太担心 - 它可能只是某种安全游戏的刷新策略:那将是对空闲实例的需求最低的时刻，因为新启动的动态实例最不可能被传入的请求淹没.

As for the apparent restarts of the resident instance right after new dynamic instances are started seems a bit odd. I wouldn't worry too much - it may simply be some sort of play-it-safe refresh strategy: that would be the moment when the need for the idle instance would be the lowest as the freshly started dynamic instance is the least likely to be overwhelmed by incoming requests.

这篇关于为什么更多的请求发送到新的(动态)实例而不是常驻实例?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！