"gcloud应用程序记录尾巴"显示一周前的数据 [英] "gcloud app logs tail" shows week old data

问题描述

我正在尝试流式传输我的Google Cloud Platform应用程序的日志.我注意到的第一件事是Google至少每秒发送大约10次健康检查，因此我必须 grep -v/health 来获取任何有用的信息.那是标准吗?

I'm trying to stream logs for my Google Cloud Platform application. The first thing I notice is that Google sends about 10 healthchecks per second, at a minimum, so I have to grep -v /health to get any useful info. Is that standard?

第二件事是它流传输旧日志.我正在运行 gcloud应用程序日志尾部-s默认值，并从4月11日(此时是一个星期)开始获取日志.较新的日志(从4、3和2天前开始)在数小时内缓慢地流进来.我只是在应用程序上点击了刷新(这会触发一条日志消息)，却没有看到新的日志.

The second thing is that it streams old logs. I'm running gcloud app logs tail -s default and getting logs from April 11 (a week old at this point). Newer logs (from 4, 3, and 2 days ago) are slowly streaming in over a matter of hours. I just hit refresh on my app (which triggers a log message) and saw no new logs.

有什么方法可以近实时(几分钟或几小时内)查看新日志消息?

Is there any way to get a close-to-realtime (within minutes or hours) view of new log messages?

推荐答案

对于运行状况检查，您无需依赖 grep .该SDK具有包含的过滤器，因此您只需过滤输出即可.但是，您只是将它们隐藏在终端上，但SDK仍会从API中获取它们(就像使用 grep 一样).更好的方法是使用 gcloud日志读取'resource.type ="gae_app" AND logName:"logs/appengine.googleapis.com％2Frequest_log" AND protoPayload.resource！="/health"'--order desc，因为您将只请求与自定义过滤器匹配的日志.这给出了一个非常详细的日志，因此您可以直接在SDK上格式化输出使其与 gcloud应用程序日志尾部类似(如果需要).

For the healthchecks, you don't need to rely in grep. The SDK has included filters, so you could just filter the output. However, you would just be hiding them on the terminal, but the SDK would still get them from the API (just like using grep). A more optimal way would be using gcloud logging read 'resource.type="gae_app" AND logName:"logs/appengine.googleapis.com%2Frequest_log" AND protoPayload.resource!="/health"' --order desc, as you would be requesting only the logs matching a custom filter. This gives quite a verbose log, so you can format the output directly on the SDK to make it as similar to gcloud app logs tailas you want.

自 gcloud日志记录 开始没有"tail"模式，您可以将其包装在 watch 中，例如:

watch 'gcloud logging read "resource.type=\"gae_app\" AND logName:\"logs/appengine.googleapis.com%2Frequest_log\" AND protoPayload.resource!=\"/health\"" --order desc --limit 1'

您应该添加-format 标志以使输出满足您的需求，并添加-limit 标志对您有意义的内容(极限，响应速度越慢).

You should add the --format flag to make the output fit your needs, and the --limit flag to something that make sense to you (the larger the limit, the slower the response).

关于日志的开始时间，如果您运行 gcloud app logs tail -s default --log-http 并将SDK完成的请求与API上可用的过滤器进行比较，则您'将了解为什么它开始显示一个星期的日志.

Regarding the time where the logs start, if you run gcloud app logs tail -s default --log-http and compare the request done by the SDK with the filters available on the API, you'll see why it starts showing one week old logs.

我认为，在您的情况下，最好的选择是直接调用包装在 watch 中的API.

I think that the best option in your case would be making the call directly to the API, wrapped in watch.

直接调用API还可让您添加自定义过滤器，并使用

A direct API call would also allow you to add a custom filter and use a field mask, returning only the relevant log entries and fields, making it less straining to the network and having faster responses.

首先，您必须创建一个日志过滤器，该过滤器将返回仅您想要的日志(无运行状况检查或监视垃圾邮件).最好的方法是在控制台本身中进行测试，直到您满意为止显示日志.

First you have to create a log filter that will return only the logs you want (no healthcheck or monitoring spam). The best way to do this is by testing in the console itself until you're satisfied with the logs displayed.

然后，检查您感兴趣的字段.对于GAE日志，很可能只需要protoPayload(并且只需要有效负载的某些字段，但可以稍后对其进行过滤).

Then you check the fields you're interested in. For GAE logs, most likely you'll want only the protoPayload (and only some fields of the payload at that, but those can be filtered later).

因此，我们以以下方式构造我们的API调用循环(警告:引用转义为地狱):

So we construct our API call loop in the following way (warning: quote escaping hell):

watch -tcn 0.5 'curl  -H"Authorization: Bearer $(gcloud auth print-access-token)" \
   -H"Content-Type: application/json" \
   "https://logging.googleapis.com/v2/entries:list?fields=entries%2FprotoPayload" \
   -d"{
   \"filter\":\"resource.type=\\\"gae_app\\\"
                logName=\\\"projects/$(gcloud config get-value project)/logs/appengine.googleapis.com%2Frequest_log\\\"\",
   \"pageSize\":$(tput lines),
   \"orderBy\":\"timestamp desc\",
   \"resourceNames\": [
      \"projects/$(gcloud config get-value project)\"
   ]
 }" 2>\dev\null |jq -cC ".entries[].protoPayload | { timestamp: .startTime, method, status, latency, URL: (.host + .resource) }"'

作为一个快速测试，我通过 jq 通过管道传递响应，以格式化输出并将响应大小限制为屏幕大小，但是您应该将其适应于字段和使其适应的输出更容易阅读.

As a quick test, I'm piping the response through jq to format the output and limiting the response size to the screen size, but you should adapt this to the fields and output that makes it easier for you to read.

这篇关于"gcloud应用程序记录尾巴"显示一周前的数据的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！