为什么对新(动态)实例的请求要比对常驻实例的请求多? [英] Why do more requests go to new (dynamic) instances than to resident instance?

问题描述

在App Engine标准环境中拥有一个具有自动缩放功能的Java应用.现在，缩放比例配置如下:

Have a Java app with auto-scaling on App Engine Standard Environment. Right now the scaling is configured like this:

<instance-class>F2</instance-class>

<automatic-scaling>
    <min-idle-instances>1</min-idle-instances>

    <!-- ‘automatic’ is the default value. -->
    <max-idle-instances>2</max-idle-instances>

    <!-- ‘automatic’ is the default value. -->
    <min-pending-latency>2000ms</min-pending-latency>

    <max-pending-latency>8000ms</max-pending-latency>
    <max-concurrent-requests>60</max-concurrent-requests>
</automatic-scaling>

只是开始尝试F2实例，而之前使用的是F1实例.无论我如何配置自动缩放，似乎新创建的实例(在负载增加时创建的实例)开始获取所有传入请求，而常驻实例的负载非常轻.

Just started trying the F2 instance, was using F1 instances earlier. No matter how I configure my auto-scaling, it seems like the newly-created instance (created when load increases) starts getting all the incoming requests, while the resident instance sits with a very light load.

这是为什么?当然，我无法实时监视流量(以及流向哪个实例)，但每次查看时，故事似乎都是一样的.我在下面提供了一些示例屏幕截图.

Why is this? Of course, I am unable to monitor the traffic (and which instance it goes to) in real time, but every time I look the story seems to be the same. I have included a few sample screenshots below.

并且在以下情况下，三个实例(与上面的实例稍有不同)处于空闲状态，但是GAE的负载平衡器选择将所有请求发送到具有最高延迟的实例！

and in the following case three instances (this was a slightly different configuration from the one above) are sitting free, but GAE's load balancer chooses to send all requests to the instance with the highest latency!

另一个例子:这是今天开始于10:15:45的 resident 实例的请求日志:

One more example: this is the request log for the resident instance started at 10:15:45 AM today:

以及10秒钟后开始的动态实例的请求日志:

and the request log for the dynamic instance that started 10 seconds later:

正如您所看到的，动态实例正在处理所有请求(到目前为止，是1889年)，而居民实际上处于空闲状态(在同一时间段中为7个).如果不是因为驻留实例似乎在创建新的动态实例的同时被销毁并重新创建 的事实，那还是可以的.这意味着在一分钟左右的时间内，所有请求的响应时间为10-20秒.

As you can see the dynamic instance is handling all of the requests (1889 so far) while the resident sits essentially idle (7 in the same time period). This would still be OK if not for the fact that the resident instances seem to be destroyed and created anew right around the time new dynamic instances are being created. This means that for a minute or so all requests see 10-20 second response times.

有人可以向我解释如何配置吗?

Can somebody please explain to me how to configure?

这就是我想要的:

• (目前)大多数情况下，一个空闲实例应该能够处理负载.
• 当有更多请求进入时，启动另一个实例. 准备就绪，开始将流量转移到该站点.

• One idle instance should be able to handle the load most of the times (for now).
• When more requests come in, spin up an additional instance. When it is ready, start diverting traffic to it.

我正试图以不菲的预算运行一个合理的负载站点，所以重要的是，我要尽量保持接近免费配额.

I am trying to run a reasonable-load site on a shoestring budget, so it is important that I try to stay as close to the free quota as possible.

由于这两个答案都突出地提到了热身请求，所以我想在这里列出有关它的详细信息.我正在使用ServletContextListener来处理初始化.它执行以下操作(使用Guava的

Since both the answers talk about the warmup request prominently, I thought I'd list details about it here. I am using a ServletContextListener to handle the initialization. It does the following (times are gathered using Guava's Stopwatch class and are for the code I have written/am explicitly invoking):

1. 注册Objectify实体(1.449秒)
2. Freemarker初始化229毫秒
3. Firebase初始化228.2 ms

除此以外，在我的web.xml中配置了Shiro过滤器，Objectify过滤器和Jersey过滤器(在Jersey中，我通过明确注册类而不是提供要扫描的包来避免类路径扫描(我认为). .不使用任何依赖项注入来避免类路径扫描.

Other than that I have the Shiro filter, the Objectify filter and Jersey filter (in Jersey I am avoiding classpath scanning (I think) by explicitly registering the classes rather than giving it a package to scan) configured in my web.xml. Not using any dependency injection to avoid classpath scanning.

/_ah/warmup请求花费了7.8s(从上述时间开始的时间).但是，由刚启动的动态实例(已完成预热)处理的请求需要10秒钟以上的时间才能完成，尽管这些调用在两分钟后需要200-700毫秒.那么，除了我在StartupListener中明确要做的事情之外，背景中还有什么其他事情?

The /_ah/warmup request took 7.8s (the one from where the above times are taken). But requests being served by a freshly-started dynamic instance whose warmup has already finished are taking 10+ seconds to complete, despite the fact that these same calls take 200-700ms two minutes later. So what else is going on in the background other than the stuff I am explicitly doing in my StartupListener?

这是日志的第1部分，这里是日志的第2部分.

推荐答案

空闲实例的作用不是处理常规流量，而是能够处理溢出-已运行流量的临时峰值动态实例(如果有)在启动新实例之前无法处理.

The role of the idle instance(s) is not to handle the usual traffic, but to be able to handle overflows - temporary peaks of traffic that the already running dynamic instances (if any) can't handle, until new instances are being started.

从某种意义上讲，这就是为什么他们被称为 idle 的原因-在大多数情况下，他们只是闲着.恕我直言，在遇到预算压力时，删除空闲实例是首先要做的事情之一.

In a sense that's why they're called idle - most of the time they're just sitting idle. IMHO dropping idle instances is one of the first things to do when on budget pressure.

也可能相关:在Google App引擎中，只有一个实例可以处理大部分请求

旁注:不是the GAE's load balancer chooses to send all requests to the instance with the highest latency!.实际上，该首选实例上的延迟是最高的，因为它是获得最多流量的一个.

Side note: it's not that the GAE's load balancer chooses to send all requests to the instance with the highest latency!. Actually the latency on that preferred instance is the highest because it's the one getting the majority of the traffic.

为防止GAE在新实例准备处理之前向其发送流量，您需要配置(并正确处理)

To prevent GAE from sending traffic to new instances before they're ready to handle them you need to configure (and properly handle) warmup requests:

预热请求是一种特定类型的加载请求 在任何活动之前，将应用程序代码提前放入实例 提出了要求.要了解有关如何使用预热请求的更多信息，请参见 预热请求.手动或基本扩展实例不接收 一个/_ah/warmup请求.

Warmup requests are a specific type of loading request that load application code into an instance ahead of time, before any live requests are made. To learn more about how to use warmup requests, see Warmup Requests. Manual or basic scaling instances do not receive an /_ah/warmup request.

您可能还想给这个答案一个想法:

You may also want to give this answer a thought: How do you keep a running instance for Google App Engine. Basically try to keep the dynamic instances running indefinitely by preventing them from remaining idle for too long via a cron job.

至于在启动新的动态实例之后立即重新启动驻留实例，似乎有些奇怪.我不会太担心-可能只是某种播放安全"的刷新策略:那是当空闲实例的需求最低时的那一刻，因为刚启动的动态实例最不可能出现这种情况.进来的请求不知所措.

As for the apparent restarts of the resident instance right after new dynamic instances are started seems a bit odd. I wouldn't worry too much - it may simply be some sort of play-it-safe refresh strategy: that would be the moment when the need for the idle instance would be the lowest as the freshly started dynamic instance is the least likely to be overwhelmed by incoming requests.

这篇关于为什么对新(动态)实例的请求要比对常驻实例的请求多?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！