Firebase托管的云功能可对任何需要60秒的请求重试，即使超时时间大于60秒也是如此 [英] Firebase-Hosted Cloud Function retrying on any request that takes 60s, even when timeout is >60s

问题描述

我有一个香草云函数，该函数需要60秒，然后使用简单的JSON对象返回状态200.该功能的超时设置为150s.在本地测试时，以及通过它的cloudfunctions.net地址运行该功能时，该功能将在60s时完成，并且200响应和主体将正确交付给客户端.到目前为止一切顺利.

这是关键-如果我运行通过Firebase托管代理的完全相同的功能(通过firebase.json内部的目标"进行设置)，根据stackdriver日志，该功能会在1-3次的任何地方立即重启，完成这些操作后，有时会重新启动该功能，最终从Varnish返回503超时.

在通过Firebase托管代理的域上调用该函数时，此行为只能始终复制.似乎仅在该功能花费约60秒或更长时间时才会发生.它不依赖于返回的响应代码或响应正文.

您可以在我在此处设置的测试功能中看到此行为: https://trellisconnect.com/testtimeout?sleepAmount=60&retCode=200

此行为最初是在通过无服务器部署的功能中确定的.为了排除无服务器，我创建了一个测试功能，该功能使测试和验证行为变得容易，并使用常规的firebase函数对其进行了部署，并从它的cloudfunctions.net域中对其进行了调用，并验证了我总是在60秒钟得到正确的响应.然后，我更新了firebase.json以添加指向该函数的新路由，并能够复制该问题.

index.js

 功能sleep(ms){返回新的Promise(resolve => setTimeout(resolve，ms));}exports.testtimeout =函数.https.onRequest((req，res)=> {const {sleepAmount，retCode} = req.query;console.log(`正在开始测试睡眠$ {sleepAmount} ...));sleep(1000 * sleepAmount).then(result => {console.log(`结束测试功能，返回$ {retCode}`);返回res.status(retCode).json({message:'Random Response'});});}); 

firebase.json

  {托管":{"public":"public"，"ignore":["firebase.json"，"**/.*"，"**/node_modules/**"]，重写":[{"source":"/testtimeout"，"function":"testtimeout"}]}，职能": {}}</snip> 

正确/预期的响应(sleepAmount = 2秒)

  zgoldberg @ zgblade:〜$时间卷曲"https://trellisconnect.com/testtimeout?sleepAmount=2&retCode=200"{"message":"Random Response"}真正的0m2.269s用户0m0.024ssys 0m0.000s 

还有一个示例，将sleepAmount设置为60秒时的外观

  zgoldberg @ zgblade:〜$ curl -v"https://trellisconnect.com/testtimeout?sleepAmount=60&retCode=200"*尝试151.101.65.195 ...* TCP_NODELAY设置*连接到trellisconnect.com(151.101.65.195)端口443(#0)* ALPN，提供h2* ALPN，提供http/1.1*成功设置证书验证位置:* CAfile:/etc/ssl/certs/ca-certificates.crtCApath:/etc/ssl/certs* TLSv1.3(OUT)，TLS握手，客户端问候(1):* TLSv1.3(IN)，TLS握手，服务器问候(2):* TLSv1.2(IN)，TLS握手，证书(11):* TLSv1.2(IN)，TLS握手，服务器密钥交换(12):* TLSv1.2(IN)，TLS握手，服务器完成(14):* TLSv1.2(OUT)，TLS握手，客户端密钥交换(16):* TLSv1.2(OUT)，TLS更改密码，客户端问候(1):* TLSv1.2(OUT)，TLS握手，已完成(20):* TLSv1.2(IN)，TLS握手，完成(20):*使用TLSv1.2/ECDHE-RSA-AES128-GCM-SHA256的SSL连接* ALPN，服务器接受使用h2*服务器证书:*主题:CN = admin.cliquefood.com.br*开始日期:2019年10月16日20:44:55 GMT*到期日期:2020年1月14日20:44:55 GMT* subjectAltName:主机"trellisconnect.com"与证书的"trellisconnect.com"匹配*发行者:C = US；O =让我们加密；CN =让我们的加密机构X3* SSL证书验证成功.*使用HTTP2，服务器支持多用途*连接状态已更改(已确认HTTP/2)*升级后将流缓冲区中的HTTP/2数据复制到连接缓冲区中:len = 0*使用流ID:1(易于处理0x563f92bdc580)>GET/testtimeout?sleepAmount = 60& retCode = 200 HTTP/2>主持人:trellisconnect.com>用户代理:curl/7.58.0>接受: */*>*连接状态已更改(已更新MAX_CONCURRENT_STREAMS个)！<HTTP/2 503<服务器:清漆<重试后:0<内容类型:text/html;字符集= utf-8<接受范围:字节<日期:2019年11月8日星期五03:12:08 GMT<x-served by:cache-bur17523-BUR<X快取:MISS<x-cache-hits:0<x计时器:S1573182544.115433，VS0，VE184552<内容长度:449<<?xml version ="1.0" encoding ="utf-8"?><！DOCTYPE html PUBLIC-//W3C//DTD XHTML 1.0 Strict//EN""http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">< html>< head>< title> 503第一字节超时</title></head><身体>< h1>错误503第一字节超时</h1>< p>第一字节超时</p>< h3>大师调解:</h3>< p>细节:cache-bur17523-BUR 1573182729 2301023220</p>< hr>< p>清漆缓存服务器</p></body></html>*与主机trellisconnect.com的连接#0保持不变真正的3m3.763s用户0m0.024sSYS 0分0.031秒 

这是疯狂的部分，检查stackdriver日志，注意该功能如何在60秒钟内完成，并且几乎在启动3个以上的执行后立即完成...

请注意，原始通话在19:09:04.235到达，并在19:10:04.428结束-几乎恰好在60s之后.几乎恰好在500毫秒之后的19:10:05.925，该功能重新启动.我向您保证，在初始响应后0.5秒内不会再次击中curl命令.此函数的后续执行都不是由我生成的，它们似乎都是幻像重试?

https://i.imgur.com/WDY17pw.png (我没有10个声誉来发布实际图像，所以只是上面的链接)

任何想法或帮助都值得赞赏

解决方案

来自 Firebase托管:使用Cloud功能为Firebase提供动态内容:

注意:Firebase托管的请求超时时间为60秒.即使您将HTTP函数配置为具有更长的请求超时时间，但是如果您的函数需要60秒钟以上的运行时间，您仍会收到HTTP状态代码 504 (请求超时).为了支持需要更长计算时间的动态内容，请考虑使用 App Engine灵活环境

简而言之，很遗憾，您的用例不受支持，因为CDN/Hosting实例仅假定连接已丢失，然后重试.

I have a vanilla cloud function that takes 60 seconds and then returns status 200 with a simple JSON object. The timeout for the function is set to 150s. When testing locally, and when running the function via it's cloudfunctions.net address, the function completes at 60s and the 200 response and body are correctly delivered to the client. So far so good.

Here's the kicker -- If I run the exact same function proxied through firebase hosting (setup via a "target" inside firebase.json), according to the stackdriver logs, the function is instantaneously restarted anywhere from 1-3 times, and when those finish the function sometimes is AGAIN restarted, eventually returning a 503 Timeout from Varnish.

This behavior is ONLY consistently replicable when the function is called on a domain that is proxied through firebase hosting. It seems to ONLY happen when the function takes ~60s or longer. It does not depend on the returned response code or response body.

You can see this behavior in a test function I have setup here: https://trellisconnect.com/testtimeout?sleepAmount=60&retCode=200

This behavior was originally identified in a function that is deployed via serverless. To rule out serverless I created a test function that makes testing and verifying the behavior easy and deployed it with regular firebase functions and called it from it's cloudfunctions.net domain and verified that I always got a correct response at 60s. I then updated my firebase.json to add a new route that points to this function and was able to replicate the problem.

index.js

function sleep(ms) {
  return new Promise(resolve => setTimeout(resolve, ms));
}

exports.testtimeout = functions.https.onRequest((req, res) => {
  const { sleepAmount, retCode } = req.query;
  console.log(`starting test sleeping ${sleepAmount}...`);
  sleep(1000 * sleepAmount).then(result => {
    console.log(`Ending test func, returning ${retCode}`);
    return res.status(retCode).json({ message: 'Random Response' });
  });
});

firebase.json

{
  "hosting": {
    "public": "public",
    "ignore": ["firebase.json", "**/.*", "**/node_modules/**"],
    "rewrites": [

      {
        "source": "/testtimeout",
        "function": "testtimeout"
      }
    ]
  },
  "functions": {}
}
</snip>

A correct/expected response (sleepAmount=2 seconds)

zgoldberg@zgblade:~$ time curl "https://trellisconnect.com/testtimeout?sleepAmount=2&retCode=200"
{"message":"Random Response"}
real    0m2.269s
user    0m0.024s
sys 0m0.000s

And a sample of how things appear when sleepAmount is set to 60 seconds

zgoldberg@zgblade:~$ curl -v "https://trellisconnect.com/testtimeout?sleepAmount=60&retCode=200"
*   Trying 151.101.65.195...
* TCP_NODELAY set
* Connected to trellisconnect.com (151.101.65.195) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=admin.cliquefood.com.br
*  start date: Oct 16 20:44:55 2019 GMT
*  expire date: Jan 14 20:44:55 2020 GMT
*  subjectAltName: host "trellisconnect.com" matched cert's "trellisconnect.com"
*  issuer: C=US; O=Let's Encrypt; CN=Let's Encrypt Authority X3
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x563f92bdc580)
> GET /testtimeout?sleepAmount=60&retCode=200 HTTP/2
> Host: trellisconnect.com
> User-Agent: curl/7.58.0
> Accept: */*
> 
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!
< HTTP/2 503 
< server: Varnish
< retry-after: 0
< content-type: text/html; charset=utf-8
< accept-ranges: bytes
< date: Fri, 08 Nov 2019 03:12:08 GMT
< x-served-by: cache-bur17523-BUR
< x-cache: MISS
< x-cache-hits: 0
< x-timer: S1573182544.115433,VS0,VE184552
< content-length: 449
< 
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
 "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html>
  <head>
    <title>503 first byte timeout</title>
  </head>
  <body>
    <h1>Error 503 first byte timeout</h1>
    <p>first byte timeout</p>
    <h3>Guru Mediation:</h3>
    <p>Details: cache-bur17523-BUR 1573182729 2301023220</p>
    <hr>
    <p>Varnish cache server</p>
  </body>
</html>
* Connection #0 to host trellisconnect.com left intact
real    3m3.763s
user    0m0.024s
sys 0m0.031s

Here's the crazy part, checkout the stackdriver logs, notice how the function completes in 60s and almost immediately after 3 more executions are started...

Notice the original call comes in at 19:09:04.235 and ends at 19:10:04.428 -- almost exactly 60s later. Almost exactly 500ms later, 19:10:05.925 the function is restarted. I promise to you I did not hit my curl command again 0.5s after the initial response. None of the subsequent exectutions of the function here were generated by me, they all seem to be phantom retries?

https://i.imgur.com/WDY17pw.png (edit: I don't have 10 reputation to post the actual image, so just a link above)

Any thoughts or help is much appreciated

解决方案

From Firebase Hosting: Serving Dynamic Content with Cloud Functions for Firebase:

Note: Firebase Hosting is subject to a 60-second request timeout. Even if you configure your HTTP function with a longer request timeout, you'll still receive an HTTP status code 504 (request timeout) if your function requires more than 60 seconds to run. To support dynamic content that requires longer compute time, consider using an App Engine flexible environment.

In short, unfortunately your use-case is not supported as the CDN/Hosting instance just assumes that the connection was lost and tries again.

这篇关于Firebase托管的云功能可对任何需要60秒的请求重试，即使超时时间大于60秒也是如此的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！